[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 503625443 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.003228] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009009] pid_max: default: 32768 minimum: 301 [ 0.010131] LSM: Security Framework initializing [ 0.011050] Yama: becoming mindful. [ 0.012006] SELinux: Initializing. [ 0.012844] *** VALIDATE selinux *** [ 0.020657] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025271] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026127] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027087] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028089] *** VALIDATE tmpfs *** [ 0.030071] *** VALIDATE proc *** [ 0.031231] *** VALIDATE cgroup *** [ 0.032008] *** VALIDATE cgroup2 *** [ 0.033236] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035048] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037026] Spectre V2 : User space: Vulnerable [ 0.038007] Speculative Store Bypass: Vulnerable [ 0.040839] debug: unmapping init [mem 0xffffffffbbe59000-0xffffffffbbe60fff] [ 0.042153] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043587] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044020] ... version: 2 [ 0.045017] ... bit width: 48 [ 0.046016] ... generic registers: 4 [ 0.047011] ... value mask: 0000ffffffffffff [ 0.048014] ... max period: 00007fffffffffff [ 0.049016] ... fixed-purpose events: 3 [ 0.050012] ... event mask: 000000070000000f [ 0.051296] rcu: Hierarchical SRCU implementation. [ 0.053467] smp: Bringing up secondary CPUs ... [ 0.054639] x86: Booting SMP configuration: [ 0.055023] .... node #0, CPUs: #1 #2 #3 [ 0.058402] smp: Brought up 1 node, 4 CPUs [ 0.060013] smpboot: Max logical packages: 1 [ 0.061018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.145230] node 0 deferred pages initialised in 80ms [ 0.148137] devtmpfs: initialized [ 0.149233] x86/mm: Memory block size: 128MB [ 0.151731] gcov: version magic: 0x41383552 [ 0.153318] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.154077] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.155224] pinctrl core: initialized pinctrl subsystem [ 0.156192] [ 0.156786] ************************************************************* [ 0.157013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.158015] ** ** [ 0.159015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.160013] ** ** [ 0.161012] ** This means that this kernel is built to expose internal ** [ 0.162014] ** IOMMU data structures, which may compromise security on ** [ 0.163014] ** your system. ** [ 0.164032] ** ** [ 0.165015] ** If you see this message and you are not debugging the ** [ 0.166016] ** kernel, report this immediately to your vendor! ** [ 0.167014] ** ** [ 0.168014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169013] ************************************************************* [ 0.170726] NET: Registered protocol family 16 [ 0.171408] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.172059] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.173055] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.174580] cpuidle: using governor menu [ 0.177288] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.178443] PCI: Using configuration type 1 for base access [ 0.179144] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.187144] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.188035] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.190093] cryptd: max_cpu_qlen set to 1000 [ 0.192224] ACPI: Added _OSI(Module Device) [ 0.193014] ACPI: Added _OSI(Processor Device) [ 0.194013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.195013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.199812] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.203193] ACPI: Interpreter enabled [ 0.204056] ACPI: PM: (supports S0 S3 S4 S5) [ 0.205013] ACPI: Using IOAPIC for interrupt routing [ 0.206098] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.207397] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.217124] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.218041] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.219020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.220075] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.222119] acpiphp: Slot [2] registered [ 0.223116] acpiphp: Slot [5] registered [ 0.224138] acpiphp: Slot [6] registered [ 0.225087] acpiphp: Slot [7] registered [ 0.226140] acpiphp: Slot [8] registered [ 0.227111] acpiphp: Slot [9] registered [ 0.228119] acpiphp: Slot [10] registered [ 0.229112] acpiphp: Slot [3] registered [ 0.230100] acpiphp: Slot [4] registered [ 0.231118] acpiphp: Slot [11] registered [ 0.232099] acpiphp: Slot [12] registered [ 0.233120] acpiphp: Slot [13] registered [ 0.234104] acpiphp: Slot [14] registered [ 0.235105] acpiphp: Slot [15] registered [ 0.236103] acpiphp: Slot [16] registered [ 0.237109] acpiphp: Slot [17] registered [ 0.238134] acpiphp: Slot [18] registered [ 0.239103] acpiphp: Slot [19] registered [ 0.241106] acpiphp: Slot [20] registered [ 0.244111] acpiphp: Slot [21] registered [ 0.246148] acpiphp: Slot [22] registered [ 0.248103] acpiphp: Slot [23] registered [ 0.249192] acpiphp: Slot [24] registered [ 0.251120] acpiphp: Slot [25] registered [ 0.253226] acpiphp: Slot [26] registered [ 0.255120] acpiphp: Slot [27] registered [ 0.257200] acpiphp: Slot [28] registered [ 0.259161] acpiphp: Slot [29] registered [ 0.261085] acpiphp: Slot [30] registered [ 0.263103] acpiphp: Slot [31] registered [ 0.264177] PCI host bridge to bus 0000:00 [ 0.266018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.269021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.271027] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.274020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.277022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.279019] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.281145] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.284538] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.288324] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.297787] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.303037] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.305016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.308018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.310017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.314257] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.317810] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.320041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.323193] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.328012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.343013] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.348901] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.354314] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.369016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.381016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.412016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.424161] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.430013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.441014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.467014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.476870] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.485013] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.495015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.513017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.526556] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.533016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.546015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.569017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.581035] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.597022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.605021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.635017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.650297] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.660014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.669014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.693017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.709711] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.712368] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.714346] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.717383] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.720210] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.723116] iommu: Default domain type: Passthrough [ 0.725560] SCSI subsystem initialized [ 0.727450] ACPI: bus type USB registered [ 0.730168] usbcore: registered new interface driver usbfs [ 0.733153] usbcore: registered new interface driver hub [ 0.734116] usbcore: registered new device driver usb [ 0.737227] pps_core: LinuxPPS API ver. 1 registered [ 0.739131] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.743106] PTP clock support registered [ 0.746083] EDAC MC: Ver: 3.0.0 [ 0.749149] PCI: Using ACPI for IRQ routing [ 0.751428] NetLabel: Initializing [ 0.753014] NetLabel: domain hash size = 128 [ 0.754013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.756093] NetLabel: unlabeled traffic allowed by default [ 0.758193] vgaarb: loaded [ 0.760136] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.762015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.771000] clocksource: Switched to clocksource kvm-clock [ 0.887821] VFS: Disk quotas dquot_6.6.0 [ 0.889237] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.891345] *** VALIDATE ramfs *** [ 0.892408] *** VALIDATE hugetlbfs *** [ 0.893627] pnp: PnP ACPI init [ 0.898506] pnp: PnP ACPI: found 6 devices [ 0.917717] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.921392] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.925567] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.929608] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.933472] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.936503] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.941408] NET: Registered protocol family 2 [ 0.945420] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.953109] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.956057] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.960407] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.963515] TCP: Hash tables configured (established 65536 bind 65536) [ 0.966467] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.969407] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.972884] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.976364] NET: Registered protocol family 1 [ 0.979921] RPC: Registered named UNIX socket transport module. [ 0.982657] RPC: Registered udp transport module. [ 0.985043] RPC: Registered tcp transport module. [ 0.987087] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.990015] NET: Registered protocol family 44 [ 0.991726] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.994131] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.996362] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.999171] PCI: CLS 0 bytes, default 64 [ 1.001118] Unpacking initramfs... [ 2.463613] debug: unmapping init [mem 0xffff8f9f7cc54000-0xffff8f9f7ffbffff] [ 2.467978] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.470816] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.474029] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.002159] Initialise system trusted keyrings [ 3.004180] Key type blacklist registered [ 3.006498] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.016592] zbud: loaded [ 3.020126] *** VALIDATE nfs *** [ 3.021477] *** VALIDATE nfs4 *** [ 3.023253] pstore: using deflate compression [ 3.028075] Platform Keyring initialized [ 3.171165] NET: Registered protocol family 38 [ 3.173612] Key type asymmetric registered [ 3.175876] Asymmetric key parser 'x509' registered [ 3.178623] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.182750] io scheduler mq-deadline registered [ 3.184630] io scheduler kyber registered [ 3.186485] io scheduler bfq registered [ 3.188396] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.191568] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.194840] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.197866] ACPI: Power Button [PWRF] [ 3.204703] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.213822] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.233038] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.243207] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.267916] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.300977] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.329692] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.335660] Non-volatile memory driver v1.3 [ 3.337283] Linux agpgart interface v0.103 [ 3.372781] virtio_blk virtio1: [vda] 155184 512-byte logical blocks (79.5 MB/75.8 MiB) [ 3.375610] vda: detected capacity change from 0 to 79454208 [ 3.395279] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.399082] vdb: detected capacity change from 0 to 1073741824 [ 3.415942] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.420160] vdc: detected capacity change from 0 to 2621440000 [ 3.439349] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.443031] vdd: detected capacity change from 0 to 2621440000 [ 3.459953] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.462912] vde: detected capacity change from 0 to 4294967296 [ 3.477944] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.481660] vdf: detected capacity change from 0 to 4294967296 [ 3.489785] libphy: Fixed MDIO Bus: probed [ 3.494491] usbcore: registered new interface driver usbserial_generic [ 3.497478] usbserial: USB Serial support registered for generic [ 3.502107] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.507061] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.508960] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.511503] mousedev: PS/2 mouse device common for all mice [ 3.516377] rtc_cmos 00:05: RTC can wake from S4 [ 3.519934] rtc_cmos 00:05: registered as rtc0 [ 3.520145] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.521891] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.521935] intel_pstate: CPU model not supported [ 3.533667] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.537184] hid: raw HID events driver (C) Jiri Kosina [ 3.539966] usbcore: registered new interface driver usbhid [ 3.542558] usbhid: USB HID core driver [ 3.543431] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.544683] drop_monitor: Initializing network drop monitor service [ 3.552727] Initializing XFRM netlink socket [ 3.556753] NET: Registered protocol family 10 [ 3.560056] Segment Routing with IPv6 [ 3.563333] NET: Registered protocol family 17 [ 3.566512] mpls_gso: MPLS GSO support [ 3.573418] RAS: Correctable Errors collector initialized. [ 3.575321] AVX version of gcm_enc/dec engaged. [ 3.576903] AES CTR mode by8 optimization enabled [ 3.663639] sched_clock: Marking stable (3663553540, 0)->(4637527075, -973973535) [ 3.668159] registered taskstats version 1 [ 3.670224] Loading compiled-in X.509 certificates [ 3.672790] zswap: loaded using pool lzo/zbud [ 3.696626] Key type big_key registered [ 3.707630] Key type encrypted registered [ 3.708956] ima: No TPM chip found, activating TPM-bypass! [ 3.710534] ima: Allocated hash algorithm: sha1 [ 3.711902] ima: No architecture policies found [ 3.713254] evm: Initialising EVM extended attributes: [ 3.715100] evm: security.selinux [ 3.716385] evm: security.ima [ 3.717826] evm: security.capability [ 3.719262] evm: HMAC attrs: 0x1 [ 3.721987] rtc_cmos 00:05: setting system clock to 2026-09-09 04:00:15 UTC (1788926415) [ 3.729912] debug: unmapping init [mem 0xffffffffbce03000-0xffffffffbcffffff] [ 3.733711] debug: unmapping init [mem 0xffffffffbbb82000-0xffffffffbbe58fff] [ 3.743359] Write protecting the kernel read-only data: 28672k [ 3.746992] debug: unmapping init [mem 0xffffffffba203000-0xffffffffba3fffff] [ 3.751550] debug: unmapping init [mem 0xffffffffbab14000-0xffffffffbabfffff] [ 3.792882] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.800547] systemd[1]: Detected virtualization kvm. [ 3.802099] systemd[1]: Detected architecture x86-64. [ 3.803554] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.833298] systemd[1]: No hostname configured. [ 3.834993] systemd[1]: Set hostname to . [ 3.836916] random: systemd: uninitialized urandom read (16 bytes read) [ 3.840648] systemd[1]: Initializing machine ID from random generator. [ 3.898617] random: ln: uninitialized urandom read (6 bytes read) [ 3.990224] random: systemd: uninitialized urandom read (16 bytes read) [ 3.993547] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.003718] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. [ 4.013648] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Reached target Local Encrypted Volumes. Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Sockets. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.715320] device-mapper: uevent: version 1.0.3 [ 4.717839] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.484167] virtio_net virtio0 ens2: renamed from eth0 [ 5.504355] random: fast init done [ 5.545946] scsi host0: ata_piix [ 5.602170] scsi host1: ata_piix [ 5.603528] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.606484] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.825117] dracut-initqueue[579]: RTNETLINK answers: File exists [ 10.242751] random: crng init done [ 10.244111] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 12.239335] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.689574] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.669725] SELinux: Disabled at runtime. [ 16.810231] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.828097] systemd[1]: Detected virtualization kvm. [ 16.831680] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 18.590694] systemd[1]: initrd-switch-root.service: Succeeded. [ 18.601210] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 18.614415] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 18.627621] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 18.647599] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 18.686317] systemd[1]: Starting Journal Service... Starting Journal Service... [ 18.703178] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ 19.199290] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 20.256783] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.508556] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.553632] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 22.079951] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 22.181427] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit)[ 26.321967] hrtimer: interrupt took 6717988 ns [** ] A start job is running for Configur…-only root support (8s / no limit)[ 26.994947] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 27.889530] NFS: Registering the id_resolver key type [ 27.897404] Key type id_resolver registered [ 27.907699] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started Login Service. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg655-server login: [ 94.735689] libcfs: loading out-of-tree module taints kernel. [ 94.767478] Key type ._llcrypt registered [ 94.769502] Key type .llcrypt registered [ 94.852317] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 114.299595] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 115.805280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 115.821492] alg: No test for adler32 (adler32-zlib) [ 117.576639] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 118.578823] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 120.304328] Key type lgssc registered [ 122.391561] Lustre: Echo OBD driver; http://www.lustre.org/ [ 138.219485] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 178.828430] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 193.626996] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 193.688792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 195.092876] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 195.142341] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 195.230679] Lustre: lustre-MDT0000: new disk, initializing [ 195.377958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.405382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 200.281196] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 214.338688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 214.444290] Lustre: 6522:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 214.495454] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 214.500811] Lustre: Skipped 1 previous similar message [ 214.625507] Lustre: lustre-MDT0001: new disk, initializing [ 214.726176] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 214.785775] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 214.796723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 219.546461] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.896926] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 234.039063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.253795] Lustre: lustre-OST0000: new disk, initializing [ 234.258706] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 234.265711] Lustre: 8461:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 234.363138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 240.176071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 240.192318] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 240.292811] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 240.822178] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 255.923890] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 256.114424] Lustre: lustre-OST0001: new disk, initializing [ 256.126990] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 256.134540] Lustre: 9533:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 256.238226] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 262.479111] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 265.289907] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 265.307177] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 265.390852] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 273.403493] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 278.897849] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 284.761760] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing check_logdir /tmp/testlogs/ [ 289.333468] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing yml_node [ 293.692685] Lustre: DEBUG MARKER: Client: 2.17.57.55 [ 296.372435] Lustre: DEBUG MARKER: MDS: 2.17.57.55 [ 298.784663] Lustre: DEBUG MARKER: OSS: 2.17.57.55 [ 300.324610] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 9 00:05:10 EDT 2026 [ 315.411501] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 316.967836] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 332.259624] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 332.261167] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 332.287332] Lustre: Skipped 3 previous similar messages [ 332.310691] Lustre: Skipped 3 previous similar messages [ 337.381229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 337.398899] Lustre: Skipped 3 previous similar messages [ 338.828781] Lustre: server umount lustre-MDT0000 complete [ 344.861681] LustreError: 7471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788926756 with bad export cookie 3006489573103746928 [ 344.866926] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 344.900504] LustreError: 7471:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 347.617268] LustreError: 10766:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 347.619200] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 347.619849] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 347.638442] LustreError: 10766:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 347.671303] Lustre: Skipped 1 previous similar message [ 349.674074] LustreError: 6530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 349.675801] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 349.698416] LustreError: 6530:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 349.704909] Lustre: Skipped 2 previous similar messages [ 354.785324] LustreError: 10306:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 354.785477] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 354.794253] LustreError: 10306:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 354.799821] Lustre: Skipped 1 previous similar message [ 359.392116] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 359.661859] Lustre: server umount lustre-MDT0001 complete [ 362.781283] Lustre: server umount lustre-OST0000 complete [ 366.313229] Lustre: server umount lustre-OST0001 complete [ 370.770728] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 380.258666] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 415.003254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 415.218224] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 415.248877] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 415.318817] Lustre: lustre-MDT0000: new disk, initializing [ 415.416380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 415.442696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 419.826205] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 431.092842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 431.210295] Lustre: 15088:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 431.287336] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 431.290568] Lustre: Skipped 1 previous similar message [ 431.381866] Lustre: lustre-MDT0001: new disk, initializing [ 431.491718] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 431.547164] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 431.559509] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 436.158360] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 446.107368] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 451.689774] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 458.641482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 458.917450] Lustre: lustre-OST0000: new disk, initializing [ 458.921800] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 458.929571] Lustre: 17056:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 459.047560] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 460.084171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 460.095179] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 460.196524] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 466.518379] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 477.061389] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 481.579227] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 481.931070] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 486.760195] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 487.011657] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 490.980325] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 490.991651] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 490.991891] LustreError: Skipped 1 previous similar message [ 491.002607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 494.751771] Lustre: server umount lustre-OST0000 complete [ 501.218403] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 501.248799] Lustre: Skipped 2 previous similar messages [ 504.894179] Lustre: server umount lustre-MDT0000 complete [ 506.340607] LustreError: 15094:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 506.360411] LustreError: 15094:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 510.438479] LustreError: 15977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 510.496776] LustreError: 15977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 511.245752] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788926923 with bad export cookie 3006489573103748454 [ 511.251604] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 511.265886] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 512.085760] Lustre: server umount lustre-MDT0001 complete [ 518.850776] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 00:08:48 (1788926928) [ 525.774433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 526.173994] LustreError: 19321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 526.240672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 531.262829] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 539.351474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 543.839476] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 551.538061] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 557.428731] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 565.975681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 575.021265] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 584.081774] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 590.710547] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 595.939164] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 595.945175] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 595.958101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 595.963507] Lustre: Skipped 3 previous similar messages [ 597.816704] Lustre: server umount lustre-MDT0000 complete [ 600.549135] LustreError: 19321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.558476] LustreError: 19321:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 600.682356] LustreError: 19303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927012 with bad export cookie 3006489573103749343 [ 600.683560] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 600.691578] LustreError: 19303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 600.928583] Lustre: server umount lustre-MDT0001 complete [ 615.163231] Lustre: server umount lustre-OST0000 complete [ 637.346662] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 647.658488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 648.010827] LustreError: 23702:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 648.088202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 648.093629] Lustre: Skipped 2 previous similar messages [ 652.288987] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 660.022314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 664.625687] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 667.377493] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 675.292475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 682.244709] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 689.627491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 689.771918] Lustre: lustre-OST0001: new disk, initializing [ 689.775970] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 689.782970] Lustre: 26015:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 691.515585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 691.524058] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 691.611927] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 695.831502] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 705.527203] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 708.843436] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 721.896241] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 721.902488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 721.921329] Lustre: Skipped 2 previous similar messages [ 721.926339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 726.441480] Lustre: server umount lustre-MDT0000 complete [ 728.555439] LustreError: 24999:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 728.587730] LustreError: 24999:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 730.041300] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927141 with bad export cookie 3006489573103750190 [ 730.051060] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 730.061918] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 730.547657] Lustre: server umount lustre-MDT0001 complete [ 743.486894] Lustre: server umount lustre-OST0000 complete [ 757.199605] Lustre: server umount lustre-OST0001 complete [ 772.162756] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 781.354099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 781.863409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 781.879386] Lustre: Skipped 3 previous similar messages [ 785.884683] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 793.849746] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 798.667144] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 802.166343] Lustre: 30183:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 808.670795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 814.740165] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 822.790878] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.992570] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 835.625232] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 838.504561] Lustre: 32061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 845.282247] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 845.293959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 845.325486] Lustre: Skipped 3 previous similar messages [ 845.337941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 845.345811] Lustre: Skipped 4 previous similar messages [ 850.100849] Lustre: server umount lustre-MDT0000 complete [ 853.729125] LustreError: 29015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927265 with bad export cookie 3006489573103751485 [ 853.743706] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 853.746238] LustreError: 29015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 853.974511] Lustre: server umount lustre-MDT0001 complete [ 868.621660] Lustre: server umount lustre-OST0000 complete [ 882.878339] Lustre: server umount lustre-OST0001 complete [ 890.786611] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 00:15:00 (1788927300) [ 920.673329] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 930.096650] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 930.493785] LustreError: 34866:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 930.529858] LustreError: 34866:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 935.607043] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 948.090177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 954.058860] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 956.858763] Lustre: 36015:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 962.897995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 968.724849] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 977.102608] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 983.217268] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 992.875330] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 997.765463] Lustre: 37897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1005.025804] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1005.049963] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1005.065044] Lustre: Skipped 3 previous similar messages [ 1005.073583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1005.083844] Lustre: Skipped 4 previous similar messages [ 1011.189296] Lustre: server umount lustre-MDT0000 complete [ 1015.017217] LustreError: 36397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927426 with bad export cookie 3006489573103752591 [ 1015.032043] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1015.034553] LustreError: 36397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1015.475969] Lustre: server umount lustre-MDT0001 complete [ 1029.607605] Lustre: server umount lustre-OST0000 complete [ 1043.095105] Lustre: server umount lustre-OST0001 complete [ 1051.339133] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 00:17:40 (1788927460) [ 1065.625732] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 1073.718931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1074.077856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1074.086093] Lustre: Skipped 7 previous similar messages [ 1077.921338] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1084.540222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1088.554709] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1091.788321] Lustre: 41198:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1099.291989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1107.379987] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1119.878307] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1129.571442] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1138.164492] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1141.801278] Lustre: 43074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1142.834550] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1142.840178] Lustre: Skipped 1 previous similar message [ 1146.341886] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1146.349723] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1146.359384] Lustre: Skipped 6 previous similar messages [ 1152.421198] Lustre: server umount lustre-MDT0000 complete [ 1155.793961] LustreError: 40029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927567 with bad export cookie 3006489573103753697 [ 1155.798230] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1155.809226] LustreError: 40029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1170.400653] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1170.625330] Lustre: server umount lustre-MDT0001 complete [ 1177.922430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1181.153337] LustreError: 3654:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8f9fd5fe0700 x1875825425512960/t0(0) o250->MGC192.168.206.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1186.875910] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1187.301255] LustreError: 41562:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1187.309931] LustreError: 41562:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 30 previous similar messages [ 1195.516313] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1196.358253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1201.433325] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1202.163935] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1202.169420] Lustre: Skipped 1 previous similar message [ 1208.945821] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1210.494497] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1217.075635] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1218.545151] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1227.240770] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1229.104441] Lustre: server umount lustre-MDT0000 complete [ 1232.887276] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927644 with bad export cookie 3006489573103754796 [ 1232.896079] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1232.906626] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1233.195807] Lustre: server umount lustre-MDT0001 complete [ 1248.496590] Lustre: server umount lustre-OST0000 complete [ 1261.720816] Lustre: server umount lustre-OST0001 complete [ 1271.171255] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 00:21:20 (1788927680) [ 1289.077030] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 1298.251063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1303.571526] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1312.066127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1316.291950] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1319.380966] Lustre: 48848:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1327.416327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1337.191851] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1346.654400] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1353.385701] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1361.736559] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1365.896673] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1377.788338] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1377.802813] Lustre: Skipped 6 previous similar messages [ 1377.827898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1377.842904] Lustre: Skipped 27 previous similar messages [ 1384.453428] Lustre: server umount lustre-MDT0000 complete [ 1388.971369] LustreError: 47676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788927800 with bad export cookie 3006489573103755615 [ 1388.987646] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1388.997288] LustreError: 47676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1389.468647] Lustre: server umount lustre-MDT0001 complete [ 1404.950126] Lustre: server umount lustre-OST0000 complete [ 1418.734242] Lustre: server umount lustre-OST0001 complete [ 1427.113479] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 00:23:56 (1788927836) [ 1429.512474] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1431.338289] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 00:24:01 (1788927841) [ 1441.414435] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 1449.058333] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 1469.897495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1470.244876] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1470.341610] Lustre: lustre-MDT0000: new disk, initializing [ 1470.428514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1474.602299] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1482.827495] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1488.109647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1488.311513] Lustre: lustre-OST0000: new disk, initializing [ 1488.315557] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1488.317574] Lustre: Skipped 1 previous similar message [ 1488.321498] Lustre: 54922:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1490.067351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1490.078240] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1490.107061] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1494.520551] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1505.535359] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1507.936400] Lustre: DEBUG MARKER: create test files [ 1547.179079] Lustre: server umount lustre-OST0000 complete [ 1552.294906] Lustre: server umount lustre-MDT0000 complete [ 1562.376483] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 1564.793470] Key type lgssc unregistered [ 1565.222665] LNet: 56557:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1565.230015] LNetError: 56557:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1565.237831] LNet: Removed LNI 192.168.206.155@tcp [ 1566.045265] Key type .llcrypt unregistered [ 1566.047433] Key type ._llcrypt unregistered [ 1590.155210] Key type ._llcrypt registered [ 1590.157330] Key type .llcrypt registered [ 1590.241254] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 1591.406626] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1591.430855] alg: No test for adler32 (adler32-zlib) [ 1592.668113] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 1593.030898] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 1594.704110] Key type lgssc registered [ 1595.927174] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1606.401645] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1606.424700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1608.192531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1613.951752] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1625.214835] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1633.815458] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1634.086704] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1641.625597] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1649.558132] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1652.470101] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1652.577141] Lustre: 59272:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1652.588333] Lustre: 59272:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1652.598174] Lustre: 59272:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1652.606569] Lustre: 59272:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1652.615071] Lustre: 59272:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1652.621505] Lustre: 59272:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1660.857421] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1665.359949] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1673.185271] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1673.193106] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1673.219763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1674.230757] Lustre: server umount lustre-OST0000 complete [ 1678.066854] Lustre: server umount lustre-MDT0000 complete [ 1686.598554] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 1688.956542] Key type lgssc unregistered [ 1689.269540] LNet: 60902:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1689.278123] LNetError: 60902:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1689.311491] LNet: Removed LNI 192.168.206.155@tcp [ 1690.073399] Key type .llcrypt unregistered [ 1690.076270] Key type ._llcrypt unregistered [ 1719.658315] Key type ._llcrypt registered [ 1719.659841] Key type .llcrypt registered [ 1719.772344] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 1721.457149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1721.533077] alg: No test for adler32 (adler32-zlib) [ 1722.744907] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 1723.000426] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 1724.672186] Key type lgssc registered [ 1726.098291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1734.092789] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1734.125142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1735.664753] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1740.593304] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1747.149612] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1753.502811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1753.760329] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1758.927890] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1760.947705] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1761.063765] Lustre: 63737:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1761.079719] Lustre: 63737:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1761.084029] Lustre: 63737:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1761.087508] Lustre: 63737:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1761.091241] Lustre: 63737:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1761.094923] Lustre: 63737:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1765.721362] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1770.559268] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1781.730755] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1781.730949] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1781.737532] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1784.791451] Lustre: server umount lustre-OST0000 complete [ 1788.396977] Lustre: server umount lustre-MDT0000 complete [ 1799.795320] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 1802.442361] Key type lgssc unregistered [ 1802.672229] LNet: 65296:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1802.679095] LNetError: 65296:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1802.710092] LNet: Removed LNI 192.168.206.155@tcp [ 1803.804750] Key type .llcrypt unregistered [ 1803.807102] Key type ._llcrypt unregistered [ 1824.050948] Key type ._llcrypt registered [ 1824.052550] Key type .llcrypt registered [ 1824.163442] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 1841.430394] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 1842.749415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1842.784783] alg: No test for adler32 (adler32-zlib) [ 1843.840580] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 1844.117538] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 1845.768383] Key type lgssc registered [ 1846.983990] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1879.562786] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1879.581912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1880.765860] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1880.792754] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1880.836551] Lustre: lustre-MDT0000: new disk, initializing [ 1880.906966] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1880.927352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1885.176659] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1896.360979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1896.568890] Lustre: 68450:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1896.634790] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1896.637407] Lustre: Skipped 1 previous similar message [ 1896.736591] Lustre: lustre-MDT0001: new disk, initializing [ 1896.867098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1896.936528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1896.967119] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1901.322768] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1909.765795] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1914.833583] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1920.948658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1921.215896] Lustre: lustre-OST0000: new disk, initializing [ 1921.221691] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1921.229094] Lustre: 70417:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1921.316828] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.207142] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 1930.350970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1930.369810] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1930.514727] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1938.971240] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1944.703471] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1944.906080] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1949.817658] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1949.996883] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1956.324654] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1956.333829] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1956.341958] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1956.347916] Lustre: Skipped 1 previous similar message [ 1961.444720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1961.458481] Lustre: Skipped 1 previous similar message [ 1966.048403] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1966.193747] Lustre: server umount lustre-OST0000 complete [ 1971.681771] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1971.691514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1971.723176] Lustre: Skipped 2 previous similar messages [ 1971.738543] Lustre: Skipped 1 previous similar message [ 1976.208618] Lustre: server umount lustre-MDT0000 complete [ 1980.838551] LustreError: 68441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788928392 with bad export cookie 3639577603273552362 [ 1980.844494] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1980.847753] LustreError: 68441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1981.175235] Lustre: server umount lustre-MDT0001 complete [ 1992.593377] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 00:33:21 (1788928401) [ 2004.550686] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 2040.009404] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 2049.203190] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 2080.237636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2080.482480] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2080.520503] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2080.606857] Lustre: lustre-MDT0000: new disk, initializing [ 2080.759629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2080.775980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2085.617467] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2098.392332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2098.704382] Lustre: 76440:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2098.831781] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2098.837829] Lustre: Skipped 1 previous similar message [ 2098.975601] Lustre: lustre-MDT0001: new disk, initializing [ 2099.183501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2099.346271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2099.377496] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2105.822093] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2118.005370] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2124.309275] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2131.686284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2131.981838] Lustre: lustre-OST0000: new disk, initializing [ 2131.985957] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2132.000783] Lustre: 78408:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2132.105078] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2139.461518] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2140.802754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2140.824281] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2141.011841] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2150.923392] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2155.412841] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2155.604707] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2159.884507] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2160.117314] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2166.766519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2166.766562] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2166.773302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2166.787706] Lustre: Skipped 1 previous similar message [ 2166.814741] LustreError: Skipped 1 previous similar message [ 2167.938366] Lustre: server umount lustre-OST0000 complete [ 2171.873766] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2171.875045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2171.887750] Lustre: Skipped 1 previous similar message [ 2171.898114] Lustre: Skipped 2 previous similar messages [ 2177.610821] Lustre: server umount lustre-MDT0000 complete [ 2181.004032] LustreError: 76475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788928592 with bad export cookie 3639577603273553244 [ 2181.008426] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2181.014359] LustreError: 76475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2181.296269] Lustre: server umount lustre-MDT0001 complete [ 2188.757457] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 00:36:38 (1788928598) [ 2194.919130] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2195.331672] LustreError: 80820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2195.355336] LustreError: 80820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2195.423196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2199.620378] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2200.546666] LustreError: 80821:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2205.667693] LustreError: 80820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2207.899774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2213.031546] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2219.880154] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2225.642780] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2231.741827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2232.023357] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2232.028018] Lustre: Skipped 1 previous similar message [ 2238.405631] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2246.257311] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2249.573915] Lustre: 83494:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 4d4ac362-a78d-4719-bf83-9a61549c153d at adminstrative request [ 2250.727527] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2250.754868] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2260.973318] Lustre: MGS: Client 4d4ac362-a78d-4719-bf83-9a61549c153d (at 0@lo) reconnecting [ 2260.983742] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x3282608bff9678ce to 0x3282608bff967b6e [ 2260.998983] Lustre: MGC192.168.206.155@tcp: Connection restored to 0@lo (at 0@lo) [ 2287.690986] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2287.866454] Lustre: lustre-OST0001: new disk, initializing [ 2287.874321] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2287.889371] Lustre: 83886:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2294.396758] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2294.897290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2294.910950] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2295.006377] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2301.948925] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2305.524432] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2305.530935] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2305.535616] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2305.535624] Lustre: Skipped 2 previous similar messages [ 2305.543938] Lustre: Skipped 1 previous similar message [ 2318.304287] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2318.390660] Lustre: server umount lustre-OST0001 complete [ 2320.902477] LustreError: 82644:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2320.935379] LustreError: 82644:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2322.404810] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2322.415287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2322.447736] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2322.454286] Lustre: Skipped 5 previous similar messages [ 2325.993574] LustreError: 82645:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2326.033324] LustreError: 82645:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2328.383855] Lustre: server umount lustre-OST0000 complete [ 2336.226691] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2340.322359] Lustre: server umount lustre-MDT0000 complete [ 2341.352719] LustreError: 80820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2341.374318] LustreError: 80820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2343.996431] LustreError: 80800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788928755 with bad export cookie 3639577603273554798 [ 2343.999841] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2344.015082] LustreError: 80800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2344.356194] Lustre: server umount lustre-MDT0001 complete [ 2353.050670] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 00:39:22 (1788928762) [ 2355.225089] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2356.729451] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 00:39:26 (1788928766) [ 2358.497195] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2360.761414] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 00:39:30 (1788928770) [ 2363.310512] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2365.639600] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 00:39:35 (1788928775) [ 2367.232803] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2369.091593] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 00:39:38 (1788928778) [ 2371.098135] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2377.800040] loop: module loaded [ 2377.827871] loop0: detected capacity change from 0 to 4294967296 [ 2377.873175] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2379.635732] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2384.391154] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2389.633454] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2421.190085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2421.247393] LustreError: 88012:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2421.255197] LustreError: 88012:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2421.272905] LustreError: 88012:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2421.287208] LustreError: 88012:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2441.520594] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 2450.668314] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 2487.847627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2488.136507] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2488.173492] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2488.361290] Lustre: lustre-MDT0000: new disk, initializing [ 2488.477189] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2488.486793] Lustre: Skipped 1 previous similar message [ 2488.507474] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2493.839891] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2504.154555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2504.277760] Lustre: 90881:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2504.589683] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2508.900624] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2517.346346] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2522.210453] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2528.177823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2528.525080] Lustre: 92848:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2534.561777] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2534.725428] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2534.876742] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2544.226410] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2549.207547] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2549.469883] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2554.777879] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2554.965517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2558.437883] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2558.441213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2558.456498] Lustre: Skipped 4 previous similar messages [ 2558.464896] Lustre: Skipped 5 previous similar messages [ 2571.232164] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2571.516535] Lustre: server umount lustre-OST0000 complete [ 2576.355070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2581.638959] Lustre: server umount lustre-MDT0000 complete [ 2584.032940] LustreError: 90892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2584.065859] LustreError: 90892:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2585.300175] LustreError: 92847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788928997 with bad export cookie 3639577603273555414 [ 2585.314594] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2585.317170] LustreError: 92847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2585.731125] Lustre: server umount lustre-MDT0001 complete [ 2597.051583] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 00:43:26 (1788929006) [ 2606.704205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2612.540229] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2617.827036] LustreError: 95263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2617.855047] LustreError: 95263:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2620.852764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2621.230744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2621.241643] Lustre: Skipped 3 previous similar messages [ 2625.645556] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2632.916665] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2639.066159] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2645.911669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2652.636979] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2659.574876] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2663.850355] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2664.019081] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2667.434189] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2667.580340] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2674.051552] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2674.285884] Lustre: lustre-OST0001: new disk, initializing [ 2674.290240] Lustre: Skipped 2 previous similar messages [ 2674.293433] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2674.297595] Lustre: Skipped 3 previous similar messages [ 2674.306500] Lustre: 98635:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2680.791974] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2681.975721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2681.982699] Lustre: Skipped 2 previous similar messages [ 2681.989863] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2682.095749] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2689.771858] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2694.465347] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2694.703637] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2699.302630] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2699.548057] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2719.308799] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2741.216879] Lustre: Failing over lustre-MDT0000 [ 2742.026781] Lustre: server umount lustre-MDT0000 complete [ 2743.050925] LustreError: 95259:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2743.072802] LustreError: 95259:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2743.784205] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2743.817813] Lustre: Skipped 5 previous similar messages [ 2760.160612] Lustre: 66673:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788929155/real 1788929155] req@00000000a178b549 x1875827234194304/t0(0) o400->MGC192.168.206.155@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788929171 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2760.176751] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2765.607276] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2765.608868] LDISKFS-fs (dm-0): recovery complete [ 2765.612945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2769.718932] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2773.970558] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2774.763292] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2774.769281] Lustre: 95258:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2775.053912] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2775.145896] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2775.488088] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2775.498267] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2775.527965] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2775.867630] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2775.885178] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2776.213896] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2776.226639] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2776.573407] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2776.590480] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 2 previous similar messages [ 2776.924200] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2776.939667] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2777.256731] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2777.274987] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2778.873368] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2778.888161] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2779.220602] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2779.234593] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2779.552175] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2779.567884] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2783.053132] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2783.066434] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2783.393167] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2783.403782] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2783.728177] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2783.733410] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2791.302581] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2791.315231] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 25 previous similar messages [ 2791.642905] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2791.656185] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2791.976109] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2791.980991] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2807.521552] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2807.542600] Lustre: 101484:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2807.869864] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2807.878779] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2808.200420] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2808.208612] LustreError: 101484:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2834.224120] Lustre: 101484:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2834.240433] Lustre: 101484:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2834.247624] Lustre: 101484:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 01d44d52-87bf-4a0e-8e1b-86c8636e238d@192.168.206.55@tcp [ 2834.257499] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2834.261125] Lustre: 101484:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000e348b8f8 x1875827216162048/t0(8589934782) o35->01d44d52-87bf-4a0e-8e1b-86c8636e238d@192.168.206.55@tcp:546/0 lens 392/0 e 0 to 0 dl 1788929256 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2834.294982] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2834.317652] Lustre: lustre-MDT0000: Denying connection for new client 01d44d52-87bf-4a0e-8e1b-86c8636e238d (at 192.168.206.55@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2834.351615] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2834.383880] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2834.391406] Lustre: Skipped 2 previous similar messages [ 2834.394150] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2834.438644] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 2834.441850] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2850.276583] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2850.287516] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2850.310455] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2850.319303] Lustre: Skipped 7 previous similar messages [ 2853.714657] Lustre: server umount lustre-OST0000 complete [ 2871.265917] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2871.438922] Lustre: server umount lustre-OST0001 complete [ 2880.475635] Lustre: server umount lustre-MDT0000 complete [ 2882.535826] LustreError: 95263:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2882.569546] LustreError: 95263:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 45 previous similar messages [ 2883.424293] LustreError: 95243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788929295 with bad export cookie 3639577603273606626 [ 2883.426917] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2883.429469] LustreError: 95243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2883.679795] Lustre: server umount lustre-MDT0001 complete [ 2891.646333] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 00:48:21 (1788929301) [ 2897.310469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2897.319431] Lustre: *** cfs_fail_loc=197, val=0*** [ 2912.868251] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2917.984817] LustreError: 103456:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.155@tcp: failed processing log, type 4: rc = -110 [ 2923.123510] Lustre: *** cfs_fail_loc=197, val=0*** [ 2923.129258] Lustre: Skipped 65 previous similar messages [ 2943.712266] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2943.732950] Lustre: Skipped 3 previous similar messages [ 2950.457613] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 2958.184349] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2960.390155] Lustre: server umount lustre-OST0000 complete [ 2967.628176] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 00:49:37 (1788929377) [ 2989.127774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2994.286561] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3004.355549] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3008.073036] LustreError: 105108:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3008.088604] LustreError: 105108:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3008.101880] LustreError: 105108:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3008.539832] Lustre: server umount lustre-MDT0000 complete [ 3047.213261] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 3057.231048] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 3088.020835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3088.205615] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3088.226482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3088.326935] Lustre: lustre-MDT0000: new disk, initializing [ 3088.430646] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3092.766133] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3102.289371] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3102.395153] Lustre: 109281:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3102.650763] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3106.170500] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3114.580355] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3120.069871] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3126.199246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3126.653499] Lustre: 111248:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3132.713867] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3136.660578] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3140.871920] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3145.019155] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3145.242232] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3149.609567] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3149.760398] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3151.844954] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3151.845528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3151.852730] LustreError: Skipped 4 previous similar messages [ 3151.863263] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3151.875375] Lustre: Skipped 6 previous similar messages [ 3151.902850] Lustre: Skipped 12 previous similar messages [ 3157.479796] Lustre: server umount lustre-OST0000 complete [ 3166.703231] Lustre: server umount lustre-MDT0000 complete [ 3167.201790] LustreError: 109287:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3167.225304] LustreError: 109287:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3170.623075] LustreError: 109273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788929582 with bad export cookie 3639577603273617350 [ 3170.632981] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3170.634470] LustreError: 109273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3170.990105] Lustre: server umount lustre-MDT0001 complete [ 3179.877140] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 00:53:09 (1788929589) [ 3187.805620] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 3189.819581] Key type lgssc unregistered [ 3190.081982] LNet: 113738:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3190.088584] LNetError: 113738:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3190.108635] LNet: Removed LNI 192.168.206.155@tcp [ 3190.767986] Key type .llcrypt unregistered [ 3190.770130] Key type ._llcrypt unregistered [ 3203.945835] Key type ._llcrypt registered [ 3203.948477] Key type .llcrypt registered [ 3204.014076] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 3219.240441] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 3220.705352] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3220.791467] alg: No test for adler32 (adler32-zlib) [ 3222.102480] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 3222.491727] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 3224.336243] Key type lgssc registered [ 3225.556798] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3264.144252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3264.173524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3265.546093] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3265.596930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3265.726551] Lustre: lustre-MDT0000: new disk, initializing [ 3265.901600] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3265.929696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3271.419444] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3282.076361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3282.449596] Lustre: lustre-OST0000: new disk, initializing [ 3282.462919] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3282.474088] Lustre: Skipped 1 previous similar message [ 3282.488493] Lustre: 117910:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3282.635704] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3290.216282] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3291.771339] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3291.781630] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3291.823471] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3302.887841] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3303.078401] Lustre: lustre-OST0001: new disk, initializing [ 3303.088210] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3303.097731] Lustre: 118773:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3303.227129] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3309.876513] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3311.730995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3311.753082] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3311.788435] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3321.826616] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3321.829242] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3321.833113] Lustre: Skipped 1 previous similar message [ 3323.879218] Lustre: server umount lustre-MDT0000 complete [ 3329.126205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3333.507070] LustreError: 117098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788929745 with bad export cookie 11735575962164457564 [ 3333.519654] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3333.536085] LustreError: 117098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3333.735196] Lustre: server umount lustre-OST0000 complete [ 3337.450417] Lustre: server umount lustre-OST0001 complete [ 3344.662435] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 00:55:54 (1788929754) [ 3366.857388] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 00:56:16 (1788929776) [ 3376.170880] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 3395.307196] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 00:56:45 (1788929805) [ 3432.306194] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 3442.570371] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 3477.274881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3477.492369] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3477.525240] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3477.610451] Lustre: lustre-MDT0000: new disk, initializing [ 3477.682352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3477.705347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3481.754477] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3491.050652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3491.157786] Lustre: 126525:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3491.224153] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3491.229318] Lustre: Skipped 1 previous similar message [ 3491.333663] Lustre: lustre-MDT0001: new disk, initializing [ 3491.473472] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3491.520746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3491.543110] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3496.257947] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3505.923787] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3512.124387] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3518.400887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3518.716662] Lustre: lustre-OST0000: new disk, initializing [ 3518.723611] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3518.731956] Lustre: 128493:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3518.867540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3525.244782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3525.280217] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3525.366123] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3525.476125] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3534.394193] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3538.898302] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3539.287336] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3543.533228] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3543.879588] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3551.217929] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3551.218808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3551.219455] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3551.219460] Lustre: Skipped 1 previous similar message [ 3551.244076] LustreError: Skipped 1 previous similar message [ 3551.283471] Lustre: Skipped 1 previous similar message [ 3553.190743] Lustre: server umount lustre-OST0000 complete [ 3561.442991] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3561.468101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3561.477611] Lustre: Skipped 2 previous similar messages [ 3564.404796] Lustre: server umount lustre-MDT0000 complete [ 3567.448432] LustreError: 126517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788929979 with bad export cookie 11735575962164458789 [ 3567.450452] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3567.467965] LustreError: 126517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3567.709742] Lustre: server umount lustre-MDT0001 complete [ 3576.342095] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 00:59:46 (1788929986) [ 3583.126314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3583.638153] LustreError: 130905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3583.654126] LustreError: 130905:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3583.728027] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3587.912824] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3589.090386] LustreError: 130906:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3594.211049] LustreError: 130905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3598.230547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3604.701793] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3613.497586] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3618.924759] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3624.972805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3625.311882] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3625.324544] Lustre: Skipped 1 previous similar message [ 3631.180294] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3637.784384] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3651.444605] Lustre: *** cfs_fail_loc=159, val=0*** [ 3667.791885] Lustre: lustre-MDT0000: Client 1161c628-5341-424b-9a3d-2db8ea48359a (at 192.168.206.55@tcp) reconnecting [ 3669.524757] Lustre: *** cfs_fail_loc=159, val=0*** [ 3669.530532] Lustre: Skipped 6 previous similar messages [ 3685.234861] Lustre: lustre-MDT0000: Client 1161c628-5341-424b-9a3d-2db8ea48359a (at 192.168.206.55@tcp) reconnecting [ 3705.317532] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3705.320365] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3705.321196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3705.342558] Lustre: Skipped 2 previous similar messages [ 3705.367134] LustreError: Skipped 1 previous similar message [ 3709.581496] Lustre: server umount lustre-OST0000 complete [ 3715.569272] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3715.577651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3715.580064] Lustre: Skipped 1 previous similar message [ 3715.595050] Lustre: Skipped 2 previous similar messages [ 3719.976855] Lustre: server umount lustre-MDT0000 complete [ 3720.673900] LustreError: 132196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3720.683149] LustreError: 132196:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3723.946129] LustreError: 133538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788930135 with bad export cookie 11735575962164459664 [ 3723.948096] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3723.960869] LustreError: 133538:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3724.366302] Lustre: server umount lustre-MDT0001 complete [ 3735.786449] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 3738.532567] Key type lgssc unregistered [ 3738.845278] LNet: 134718:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3738.854856] LNetError: 134718:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3738.873046] LNet: Removed LNI 192.168.206.155@tcp [ 3739.970791] Key type .llcrypt unregistered [ 3739.975582] Key type ._llcrypt unregistered [ 3764.917114] Key type ._llcrypt registered [ 3764.919893] Key type .llcrypt registered [ 3765.045531] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 3766.079060] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3766.142442] alg: No test for adler32 (adler32-zlib) [ 3767.407532] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 3767.866817] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 3769.560204] Key type lgssc registered [ 3770.890911] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3778.622348] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3778.655307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3780.239665] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3785.478235] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3789.793449] LustreError: 136133:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3789.827398] LustreError: 136133:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3794.336523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3794.663995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3799.319675] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3806.433953] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3812.127614] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3818.055589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3818.382029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3823.867453] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 3824.573121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3831.698283] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3840.908829] Lustre: *** cfs_fail_loc=159, val=0*** [ 3857.173351] Lustre: lustre-MDT0000: Client 4f748837-365e-442a-9909-e655f9538c45 (at 192.168.206.55@tcp) reconnecting [ 3859.508593] Lustre: *** cfs_fail_loc=159, val=0*** [ 3875.087094] Lustre: lustre-MDT0001: Client 4f748837-365e-442a-9909-e655f9538c45 (at 192.168.206.55@tcp) reconnecting [ 3876.550157] Lustre: *** cfs_fail_loc=159, val=0*** [ 3876.558488] Lustre: Skipped 3 previous similar messages [ 3892.013585] Lustre: lustre-MDT0001: Client 4f748837-365e-442a-9909-e655f9538c45 (at 192.168.206.55@tcp) reconnecting [ 3908.897903] Lustre: 139193:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3912.014813] Lustre: *** cfs_fail_loc=159, val=0*** [ 3912.018359] Lustre: Skipped 3 previous similar messages [ 3928.334564] Lustre: lustre-MDT0000: Client 585dacd3-6e25-474a-aefa-d2e48a7d8c9d (at 192.168.206.55@tcp) reconnecting [ 3930.176871] Lustre: *** cfs_fail_loc=159, val=0*** [ 3930.182700] Lustre: Skipped 13 previous similar messages [ 3946.801373] Lustre: lustre-MDT0000: Client 585dacd3-6e25-474a-aefa-d2e48a7d8c9d (at 192.168.206.55@tcp) reconnecting [ 3948.296218] Lustre: 139486:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3954.151542] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3954.156713] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3954.174376] LustreError: Skipped 1 previous similar message [ 3954.194035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3957.734306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3957.735910] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3957.736573] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3957.736578] LustreError: Skipped 1 previous similar message [ 3959.431588] Lustre: server umount lustre-OST0000 complete [ 3963.875570] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3963.881481] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3963.889844] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3963.898591] Lustre: Skipped 1 previous similar message [ 3967.970338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3967.974372] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3970.137626] Lustre: server umount lustre-MDT0000 complete [ 3973.089279] LustreError: 139005:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3973.102692] LustreError: 139005:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3974.124703] LustreError: 139069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788930385 with bad export cookie 4517674795703783550 [ 3974.133319] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3974.147183] LustreError: 139069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3974.646158] Lustre: server umount lustre-MDT0001 complete [ 3987.000783] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 3990.265838] Key type lgssc unregistered [ 3990.590966] LNet: 140541:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3990.605224] LNetError: 140541:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3990.622049] LNet: Removed LNI 192.168.206.155@tcp [ 3991.477309] Key type .llcrypt unregistered [ 3991.492602] Key type ._llcrypt unregistered [ 4017.010672] Key type ._llcrypt registered [ 4017.017624] Key type .llcrypt registered [ 4017.151196] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4018.700669] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4018.785336] alg: No test for adler32 (adler32-zlib) [ 4019.955803] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 4020.321937] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 4022.032174] Key type lgssc registered [ 4023.523887] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4032.240429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4032.265610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4033.888657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4039.030777] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4043.243601] LustreError: 141959:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4043.269859] LustreError: 141959:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4047.943034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4048.399499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4053.522752] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4061.913960] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4068.807633] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4075.621354] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4076.065864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4079.169383] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4082.369967] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4091.141811] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4093.509326] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4100.897870] Lustre: 144542:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4105.428728] Lustre: 144591:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4107.140617] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4108.082206] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4112.477300] Lustre: 144739:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4113.472084] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4114.659927] Lustre: 144847:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4120.039043] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4120.047606] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4120.069794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4124.129161] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4124.130410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4124.140781] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4124.171439] Lustre: Skipped 1 previous similar message [ 4126.206409] Lustre: server umount lustre-OST0000 complete [ 4130.273325] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4130.279837] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4130.299753] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4134.378847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4134.389494] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4136.488067] Lustre: server umount lustre-MDT0000 complete [ 4139.489697] LustreError: 141960:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4139.502951] LustreError: 141960:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4140.533963] LustreError: 141941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788930552 with bad export cookie 17644306659320299351 [ 4140.555807] LustreError: 141941:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4140.556850] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4141.044671] Lustre: server umount lustre-MDT0001 complete [ 4152.931836] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 4156.556273] Key type lgssc unregistered [ 4156.931683] LNet: 145911:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4156.939899] LNetError: 145911:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4156.951798] LNet: Removed LNI 192.168.206.155@tcp [ 4157.987243] Key type .llcrypt unregistered [ 4157.990311] Key type ._llcrypt unregistered [ 4184.065617] Key type ._llcrypt registered [ 4184.068645] Key type .llcrypt registered [ 4184.199966] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4185.611927] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4185.711384] alg: No test for adler32 (adler32-zlib) [ 4186.918720] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 4187.210753] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 4188.864162] Key type lgssc registered [ 4190.115783] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4197.878641] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4197.917077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4199.462115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4203.251217] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4209.644027] LustreError: 147326:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4209.655606] LustreError: 147326:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4211.224914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4211.671523] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4216.690453] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4224.760929] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4230.604186] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4237.545814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4238.000261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4245.191962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4245.241377] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4254.416923] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4256.437020] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4261.088813] Lustre: *** cfs_fail_loc=159, val=0*** [ 4277.584315] Lustre: lustre-MDT0000: Client 74ea5dc9-2b0b-46bd-b600-745c57126ad9 (at 192.168.206.55@tcp) reconnecting [ 4286.433630] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4286.445332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4286.463494] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4287.461073] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4287.471766] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4287.474521] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4289.655489] Lustre: server umount lustre-OST0000 complete [ 4293.602632] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4293.612078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4293.623957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4293.629051] Lustre: Skipped 1 previous similar message [ 4297.705522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4297.707032] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4299.622992] Lustre: server umount lustre-MDT0000 complete [ 4302.818036] LustreError: 147327:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4302.859117] LustreError: 147327:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4303.618504] LustreError: 147307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788930715 with bad export cookie 163690034546237171 [ 4303.633870] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4304.056552] Lustre: server umount lustre-MDT0001 complete [ 4315.509943] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 4318.427343] Key type lgssc unregistered [ 4318.795899] LNet: 150993:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4318.807328] LNetError: 150993:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4318.818853] LNet: Removed LNI 192.168.206.155@tcp [ 4319.792745] Key type .llcrypt unregistered [ 4319.798401] Key type ._llcrypt unregistered [ 4346.056297] Key type ._llcrypt registered [ 4346.059024] Key type .llcrypt registered [ 4346.175905] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4347.188902] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4347.280761] alg: No test for adler32 (adler32-zlib) [ 4348.362488] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 4348.650586] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 4350.336157] Key type lgssc registered [ 4351.579559] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4359.399515] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4359.413696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4361.072647] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4365.298356] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4370.403113] LustreError: 152413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4370.418610] LustreError: 152413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4373.219304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4373.649953] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4378.502889] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4386.328904] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4392.090325] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4398.330738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4398.877081] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4399.926201] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4405.686478] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4406.079260] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4414.410311] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4435.008765] Lustre: lustre-MDT0001: evicting 83478a5e-d992-4a66-b2de-81ae594ff17d (at 192.168.206.55@tcp) by administrative request [ 4435.022796] Lustre: Skipped 1 previous similar message [ 4466.658432] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4466.669391] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4466.689581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4467.682757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4467.683573] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4467.703382] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4469.747771] Lustre: server umount lustre-OST0000 complete [ 4475.874265] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4475.889095] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4475.902176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4475.912214] Lustre: Skipped 1 previous similar message [ 4477.926091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4477.928393] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4483.041831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4483.049901] Lustre: Skipped 1 previous similar message [ 4488.160400] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4488.405235] Lustre: server umount lustre-MDT0000 complete [ 4492.258915] LustreError: 152413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4492.296493] LustreError: 152413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4492.328902] LustreError: 152392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788930904 with bad export cookie 16739720205467739078 [ 4492.339597] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4492.344383] LustreError: 152392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4492.801190] Lustre: server umount lustre-MDT0001 complete [ 4504.399978] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 4507.630343] Key type lgssc unregistered [ 4508.026453] LNet: 156487:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4508.032783] LNetError: 156487:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4508.054211] LNet: Removed LNI 192.168.206.155@tcp [ 4509.076558] Key type .llcrypt unregistered [ 4509.080084] Key type ._llcrypt unregistered [ 4581.095327] Key type ._llcrypt registered [ 4581.099315] Key type .llcrypt registered [ 4581.254407] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4582.664438] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4582.760412] alg: No test for adler32 (adler32-zlib) [ 4583.913295] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 4584.151903] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 4585.897179] Key type lgssc registered [ 4587.270360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4595.908677] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4595.953655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4597.652581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4602.679981] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4607.975397] LustreError: 158977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4607.998646] LustreError: 158977:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4612.014673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4612.627838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4617.617382] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4625.014945] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4630.772365] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4637.710279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4638.112870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4639.177831] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4643.252231] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4643.911589] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4650.624657] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4659.682951] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4659.689856] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4659.711868] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4661.074853] Lustre: server umount lustre-OST0000 complete [ 4668.899926] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4668.902688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4668.933463] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4668.950980] Lustre: Skipped 1 previous similar message [ 4674.017697] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4674.031281] Lustre: Skipped 1 previous similar message [ 4675.129200] Lustre: server umount lustre-MDT0000 complete [ 4678.707448] LustreError: 158957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788931090 with bad export cookie 9841608707492144869 [ 4678.717085] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4678.720476] LustreError: 158957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4679.147949] Lustre: server umount lustre-MDT0001 complete [ 4691.279420] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 4693.905330] Key type lgssc unregistered [ 4694.314223] LNet: 162524:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4694.329471] LNetError: 162524:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4694.354836] LNet: Removed LNI 192.168.206.155@tcp [ 4695.217242] Key type .llcrypt unregistered [ 4695.218725] Key type ._llcrypt unregistered [ 4717.329428] Key type ._llcrypt registered [ 4717.332640] Key type .llcrypt registered [ 4717.424719] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4718.486847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4718.500180] alg: No test for adler32 (adler32-zlib) [ 4719.528691] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 4719.729789] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 4721.400143] Key type lgssc registered [ 4722.457107] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4735.723270] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 01:19:05 (1788931145) [ 4750.538311] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 4758.835429] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4789.685491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4789.715214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4791.071446] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4791.101974] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4791.186292] Lustre: lustre-MDT0000: new disk, initializing [ 4791.319804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4791.335409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4796.112486] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4806.309358] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4806.458469] Lustre: 166786:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4806.531782] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4806.541527] Lustre: Skipped 1 previous similar message [ 4806.646176] Lustre: lustre-MDT0001: new disk, initializing [ 4806.778182] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4806.833979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4806.849773] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4811.308747] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4820.183641] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4825.086484] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4831.338552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4831.568452] Lustre: lustre-OST0000: new disk, initializing [ 4831.573340] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4831.583083] Lustre: 168751:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4831.673428] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4838.306753] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4840.035339] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4840.041694] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4840.162314] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4847.839544] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4860.903225] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4860.904929] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4860.910288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4860.915698] LustreError: Skipped 1 previous similar message [ 4860.939240] Lustre: Skipped 1 previous similar message [ 4863.381281] Lustre: server umount lustre-OST0000 complete [ 4868.065778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4868.075756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4868.087722] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4868.095765] Lustre: Skipped 1 previous similar message [ 4871.139106] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4871.139397] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4876.258866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4876.266367] Lustre: Skipped 2 previous similar messages [ 4880.353375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4880.360963] Lustre: Skipped 1 previous similar message [ 4881.376664] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4881.614151] Lustre: server umount lustre-MDT0000 complete [ 4885.233322] LustreError: 166775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788931297 with bad export cookie 575895335353262621 [ 4885.243491] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4885.250890] LustreError: 166775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4885.557904] Lustre: server umount lustre-MDT0001 complete [ 4895.863328] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 4898.377582] Key type lgssc unregistered [ 4898.634676] LNet: 170660:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4898.641232] LNetError: 170660:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4898.661850] LNet: Removed LNI 192.168.206.155@tcp [ 4899.424730] Key type .llcrypt unregistered [ 4899.427040] Key type ._llcrypt unregistered [ 4919.135433] Key type ._llcrypt registered [ 4919.137278] Key type .llcrypt registered [ 4919.260039] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 4920.312304] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4920.329872] alg: No test for adler32 (adler32-zlib) [ 4921.415162] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 4921.645319] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 4923.312765] Key type lgssc registered [ 4924.332908] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4931.431293] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4931.442435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4932.863640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4935.827434] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4942.309822] LustreError: 172052:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4942.322084] LustreError: 172052:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4943.174987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4943.479966] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4947.048696] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4954.630562] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4959.930683] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4966.203428] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4966.629565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4971.936078] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 4977.916452] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4978.081445] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4984.483382] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4984.504316] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4984.522614] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4984.528559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.534877] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4985.400800] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4985.425837] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4985.442732] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4985.457895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.473706] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4986.378952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.145552] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4987.152389] Lustre: Skipped 1 previous similar message [ 4987.167322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4987.182618] Lustre: Skipped 1 previous similar message [ 4987.187949] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4987.193785] Lustre: Skipped 1 previous similar message [ 4987.204843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.219922] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4987.232382] Lustre: Skipped 1 previous similar message [ 4988.046173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.959666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.746343] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4989.750678] Lustre: Skipped 2 previous similar messages [ 4989.761850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4989.768081] Lustre: Skipped 2 previous similar messages [ 4989.770865] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4989.774578] Lustre: Skipped 2 previous similar messages [ 4989.777440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.786471] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4989.792928] Lustre: Skipped 2 previous similar messages [ 4990.610199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.524302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.393080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.400100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.201344] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4994.206391] Lustre: Skipped 4 previous similar messages [ 4994.217340] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4994.224198] Lustre: Skipped 4 previous similar messages [ 4994.227963] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4994.233509] Lustre: Skipped 4 previous similar messages [ 4994.240106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.253399] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4994.261318] Lustre: Skipped 4 previous similar messages [ 4995.193410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.065161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.105656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.995759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.932842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.738820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.721418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.604554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.469314] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5002.473842] Lustre: Skipped 8 previous similar messages [ 5002.487295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5002.494635] Lustre: Skipped 8 previous similar messages [ 5002.497558] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5002.501868] Lustre: Skipped 8 previous similar messages [ 5002.505915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.519148] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5002.523525] Lustre: Skipped 8 previous similar messages [ 5003.367960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.258967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.059213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.819936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.600540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.413970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.246381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.135166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.789444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.686258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.343933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.232066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.190704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.096570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.960812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.909638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.706988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.610150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.433471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.128679] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5019.141703] Lustre: Skipped 19 previous similar messages [ 5019.175181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5019.185725] Lustre: Skipped 19 previous similar messages [ 5019.198534] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5019.209254] Lustre: Skipped 19 previous similar messages [ 5019.219229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.243276] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5019.255674] Lustre: Skipped 19 previous similar messages [ 5020.402309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.442888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.674343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.506987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.429272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.398896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.267786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.286099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.248074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.141310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.061842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.974839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.883430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.802433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.783627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.802984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.687235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.628937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.449505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.469845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.419536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.376379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.152416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.117968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.846198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.749980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.600747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.318869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.161386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.881901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.763423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.474911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.169367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.085986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.936798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.788526] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5051.792860] Lustre: Skipped 35 previous similar messages [ 5051.820594] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5051.831859] Lustre: Skipped 35 previous similar messages [ 5051.835846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5051.840272] Lustre: Skipped 35 previous similar messages [ 5051.843262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.850754] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5051.854490] Lustre: Skipped 35 previous similar messages [ 5052.494520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.216904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.939659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.829870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.637861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.739986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.787904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.858159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.974078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.915363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.678439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.874484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.850219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.049464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.955989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.942028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.950071] LustreError: 172072:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5067.960329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.961531] LustreError: 172072:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5069.106528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.300503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.261067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.482828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.351590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.201499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.908367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.744483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.490067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.336940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.095729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.943903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.736449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.507169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.503663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.420516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.332382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.169390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.067775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.774217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.809860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.833764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.785667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.764922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.765506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.795697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.611133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.536470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.416757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.268404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.086392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.009742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.836542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.569660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.178219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.927377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.724760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.643072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.476257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.203471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.879859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.655524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.462265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.153141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.868861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.628073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.475740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.328415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.138193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.316825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.223693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.316147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.395492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.278496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.373435] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5116.382640] Lustre: Skipped 71 previous similar messages [ 5116.405833] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5116.417632] Lustre: Skipped 71 previous similar messages [ 5116.422362] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5116.426653] Lustre: Skipped 71 previous similar messages [ 5116.430620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.441473] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5116.448595] Lustre: Skipped 71 previous similar messages [ 5117.284301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.316808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.142538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.047757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.745699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.777058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.719832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.614156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.548076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.748351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.846091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.071145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.978182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.922645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.794421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.726755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.396490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.335402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.227199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.939053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.586460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.585734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.405644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.278046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.001816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.849150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.546510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.289795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.108915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.840744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.529747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.385205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.193987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.047248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.026754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.883965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.666917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.553482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.431728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.363295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.218069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.995676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.710995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.422831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.250615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.933970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.662431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.382690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.125984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.782728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.512566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.395685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.935057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.624765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.182658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.869228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.751914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.702539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.704637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.597801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.571928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.850813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.973895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.761473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.506834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.303720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.471860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.530052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.654308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.477366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.585688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.665173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.113308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.499333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.457618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.288109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.150619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.193224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.328884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.241737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.409209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.305793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.082995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.859187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.819443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.046590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.879167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.619375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.390956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.148111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.819231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.606336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.392162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.086247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.829204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.483560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.271888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.094564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.972355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.877407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.632159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.419774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.208390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.028906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.761066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.515630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.273964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.122307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.029478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.830330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.686732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.721153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.593125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.497798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.530133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.447733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.526634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.335106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.200719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.068915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.022976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.760171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.637768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.553609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.434088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.309354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.146955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.912704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.813789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.720902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.640706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.442611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.169750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.904177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.780199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.608295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.703432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.739951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.577109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.495057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.409917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.286742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.260354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.172176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.067197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.905696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.882286] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5244.889249] Lustre: Skipped 146 previous similar messages [ 5244.909772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5244.920374] Lustre: Skipped 146 previous similar messages [ 5244.926704] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5244.937986] Lustre: Skipped 146 previous similar messages [ 5244.950929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.962683] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5244.969688] Lustre: Skipped 146 previous similar messages [ 5245.875391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.805261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.539787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.424380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.337810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.224464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.057778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.897477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.718487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.568383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.419828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.345885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.097735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.041337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.921733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.802040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.791920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.654349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.498570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.468698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.415314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.335312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.327689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.149065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.199689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.180676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.154850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.116733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.004923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.990819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.891954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.892203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.853104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.877547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.775121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.776239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.544917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.394262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.151862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.927405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.740549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.490845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.307977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.261587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.104124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.947896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.738989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.649373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.475709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.301652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.150847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.908499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.694286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.703667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.722596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.682251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.573049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.382482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.291959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.029811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.755219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.621512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.501365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.252388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.182043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.985985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.868917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.705795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.662150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.605726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.492060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.303646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.165648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.011464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.868390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.669488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.458076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.359537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.153477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.989046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.767737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.619678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.490987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.366309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.262606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.328874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.157707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.017618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.880870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.612024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.348229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.105203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.838685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.589957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.252506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.048665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.726191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.399552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.031860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.662582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.402277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.216342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.059215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.951625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.719145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.553984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.236685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.989922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.740543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.561331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.343672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.139606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.009574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.875220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.691608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.555223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.377478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.195290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.056737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.864292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.799646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.627833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.493341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.299318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.152271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.954840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.654748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.418563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.237038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.031551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.891275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.686369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.468982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.322726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.297301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.174776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.949842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.789498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.601732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.606515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.369984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.236969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.033820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.050684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.989214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.765760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.693979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.470637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.367660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.214928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.128526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.097597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.950821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.920386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.961483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.658593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.502719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.293140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.196572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.021647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.875657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.645875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.404866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.132867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.891172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.685764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.541643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.382479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.359053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.157580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.958596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.839924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.670623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.417825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.355848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.174662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.887913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.613486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.372035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.130352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.996932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.735238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.641271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.402211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.368553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.267659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.985675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.755328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.569519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.274037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.213552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.918886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.815684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.621304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.398776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.214525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.114368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.729869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.404659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.192356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.914175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.631606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.402903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.192986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.115931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.971908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.673138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.656918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.551533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.511707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.531911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.197549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.998512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.809420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.822353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.880601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.819539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.508760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.243636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.079564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.731929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.431959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.982648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.627596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.372712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.208958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.840816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.521516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.186750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.112978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.842733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.690515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.452237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.117260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.734734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.574690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.145688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.784793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.357905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.123123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.964465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.914220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.528620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.240615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.021244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.841662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.631945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.487128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.259379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.970118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.685754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.587972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.378115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.227954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.134111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.960087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.793597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.529880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.262296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.032776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.796393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.567758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.223064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.015773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.704892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.399977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.162557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.874787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.611076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.207149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.961656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.657436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.366482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.212431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.889669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.562489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.190523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.941546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.655619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.367853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.129840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.859248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.565461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.313746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.092813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.885116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.645478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.435259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.192686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.057869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.854427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.764534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.460466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.216051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.055486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.826339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.603953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.415735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.296619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.079568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.946995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.825352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.651876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.396111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.166268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.766327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.581095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.286160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.008214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.765297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.545540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.198618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.003675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.662347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.365851] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5501.378245] Lustre: Skipped 314 previous similar messages [ 5501.394421] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5501.401308] Lustre: Skipped 314 previous similar messages [ 5501.405075] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5501.411702] Lustre: Skipped 314 previous similar messages [ 5501.420023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.427706] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5501.436966] Lustre: Skipped 314 previous similar messages [ 5502.089687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.819080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.694637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.463036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.279484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.020266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.820495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.656581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.339145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.211736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.097896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.977740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.772687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.703167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.472545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.252858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.978836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.901662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.764386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.626599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.472718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.312947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.084819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.863227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.614875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.541298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.415537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.243720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.308784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.165260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.066833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.941293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.791746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.753809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.534085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.397987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.237780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.134518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.928203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.740049] LustreError: 172072:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5534.753691] LustreError: 172072:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5534.772617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.713766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.431628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.242717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.138847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.952738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.890845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.879542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.639694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.417437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.366589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.189861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.075290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.802738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.620255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.369651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.161441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.051230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.815921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.555458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.493290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.349710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.141491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.947980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.752277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.530388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.298666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.055396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.706181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.448825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.264801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.104720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.845863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.563864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.318190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.011447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.657548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.400140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.119350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.772569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.466319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.116584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.719219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.354307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.085830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.777313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.403784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.086285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.807941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.505823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.229086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.884076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.615141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.386286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.076549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.698494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.367943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.044819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.692578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.284769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.016404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.690233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.525401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.283596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.182464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.963346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.658453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.390322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.158404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.863554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.686878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.500497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.313243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.143928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.862317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.511337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.193989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.917163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.678077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.473470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.374167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.325728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.286122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.169444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.129087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.103793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.009068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.940307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.015894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.924249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.790201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.710119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.619721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.535104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.282467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.053715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.101544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.045881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.081929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.798308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.592717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.350760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.252986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.946840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.713789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.349864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.132227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.921646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.687451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.368889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.101039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.867424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.667360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.354600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.322677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.138202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.991718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.700469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.440548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.298280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.099541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.755186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.645942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.422729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.135714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.837255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.538906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.120346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.762505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.339214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.979421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.574789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.206576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.697811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.239912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.735683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.298333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.817760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.452588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.057482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.631152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.252940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.836207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.369295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.096559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.789631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.485931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.011894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.824645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.589899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.269894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.840846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.644199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.328955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.060959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.808692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.437772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.121905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.928685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.651758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.574366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.318890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.116884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.862565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.617514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.309089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.046553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.732301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.350770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.964041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.616805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.398323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.268377] LustreError: 172072:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5664.283808] LustreError: 172072:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5664.286512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.011745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.689925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.362520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.207290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.977543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.709633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.400208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.110481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.719056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.280205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.778506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.299947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.954510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.695143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.517539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.204409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.875978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.477274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.141492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.804059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.350613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.021496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.767893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.410660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.105301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.858815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.444627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.244432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.939087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.695594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.295247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.154461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.905407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.605695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.343626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.028407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.711427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.415766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.197028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.828426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.615685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.314487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.083644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.890028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.647960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.467278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.314400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.977539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.822881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.504610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.300760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.044887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.759246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.432893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.083487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.732822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.411510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.158100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.809291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.565446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.304408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.123806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.776890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.632921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.472807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.229814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.952768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.783213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.502119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.276451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.034155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.844564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.577851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.274950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.054291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.888051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.635695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.392249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.088633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.925123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.702993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.410966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.061943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.819441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.524528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.352473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.134761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.972645] LustreError: 172072:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5727.981737] LustreError: 172072:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5727.997301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.878930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.803127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.577651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.411382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.186299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.072985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.960848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.774148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.523442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.292671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.029036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.743935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.419812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.122482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.794395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.550053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.270696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.080458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.811793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.591594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.334636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.043602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.807430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.537208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.234481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.904674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.694228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.546563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.376614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.406563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.169244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.850478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.679939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.422766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.225607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.990190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.852364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.645735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.471814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.067403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.763162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.457924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.209086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.874702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.708326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.468812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.234082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.934588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.751439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.564823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.371295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.155273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.970400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.696106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.641250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.478571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.254605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.296847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.188314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.976240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.772939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.685204] LustreError: 172072:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5776.696788] LustreError: 172072:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5776.717826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.495821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.548369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.345849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.258552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.126831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.991521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.988591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.943219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.624867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.360107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.011470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.780135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.521577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.364164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.259342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.995458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.714339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.490881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.051466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.653302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.356614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.970757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.587275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.200240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.760948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.453946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.122165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.691466] LustreError: 173873:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6110.704024] LustreError: 172072:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6495.726369] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6495.737095] Lustre: Skipped 389 previous similar messages [ 6495.747309] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6498.278846] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6498.287324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6500.852894] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6500.863117] Lustre: Skipped 1 previous similar message [ 6501.998215] Lustre: server umount lustre-OST0000 complete [ 6505.954506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6505.962286] Lustre: Skipped 1 previous similar message [ 6511.073684] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6511.082429] Lustre: Skipped 1 previous similar message [ 6512.102866] Lustre: server umount lustre-MDT0000 complete [ 6515.809201] LustreError: 172033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788932927 with bad export cookie 16964347451160748559 [ 6515.810278] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6515.818692] LustreError: 172033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6516.138421] Lustre: server umount lustre-MDT0001 complete [ 6528.597392] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 6531.944814] Key type lgssc unregistered [ 6532.258218] LNet: 227226:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6532.284544] LNetError: 227226:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6532.302878] LNet: Removed LNI 192.168.206.155@tcp [ 6533.304251] Key type .llcrypt unregistered [ 6533.305527] Key type ._llcrypt unregistered [ 6559.721379] Key type ._llcrypt registered [ 6559.723361] Key type .llcrypt registered [ 6559.857550] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 6560.968461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6561.157220] alg: No test for adler32 (adler32-zlib) [ 6562.236629] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 6562.559385] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 6564.280230] Key type lgssc registered [ 6565.702622] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6572.929621] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6572.943649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6574.333790] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6578.421942] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6584.804179] LustreError: 228647:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6584.821669] LustreError: 228647:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6585.839407] LustreError: 228648:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6587.948630] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6588.587742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6594.029920] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6602.148920] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6608.774377] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6615.891451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6616.245493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6620.354931] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6623.491839] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6631.958506] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6640.020751] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6642.461814] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6642.485155] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6642.492301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.501728] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6642.504788] LustreError: 228667:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6649.097493] Lustre: lustre-OST0000: Not available for connect from 192.168.206.55@tcp (stopping) [ 6649.318224] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6649.319586] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6649.352116] Lustre: Skipped 1 previous similar message [ 6651.215248] Lustre: server umount lustre-OST0000 complete [ 6662.585327] LustreError: 228667:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6686.177386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6686.507341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6689.593507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.631336] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.652527] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6689.658942] Lustre: Skipped 1 previous similar message [ 6693.716805] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6701.712451] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6703.478392] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6709.507215] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c9e48db3000.ost_server_uuid 50 [ 6711.027880] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c9e48db3000.ost_server_uuid in IDLE state after 0 sec [ 6716.387375] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6716.399937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6716.430887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6716.439509] Lustre: Skipped 2 previous similar messages [ 6720.486288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6720.487106] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6721.889172] Lustre: server umount lustre-OST0000 complete [ 6725.607075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6725.621424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6725.622826] Lustre: Skipped 1 previous similar message [ 6725.640267] Lustre: Skipped 2 previous similar messages [ 6730.721274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6731.710265] Lustre: server umount lustre-MDT0000 complete [ 6735.844370] LustreError: 228647:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6735.884720] LustreError: 228647:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6736.251838] LustreError: 228628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788933148 with bad export cookie 8406417058892838197 [ 6736.256434] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6736.263945] LustreError: 228628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6736.678965] Lustre: server umount lustre-MDT0001 complete [ 6749.755556] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 6753.296283] Key type lgssc unregistered [ 6753.659027] LNet: 233861:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6753.678054] LNetError: 233861:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6753.702202] LNet: Removed LNI 192.168.206.155@tcp [ 6754.541373] Key type .llcrypt unregistered [ 6754.544478] Key type ._llcrypt unregistered [ 6789.447777] Key type ._llcrypt registered [ 6789.456228] Key type .llcrypt registered [ 6789.565499] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 6790.734719] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6791.089490] alg: No test for adler32 (adler32-zlib) [ 6792.203326] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 6792.551830] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 6794.288213] Key type lgssc registered [ 6795.998559] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6806.290434] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6806.310403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6807.775643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6812.436272] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6817.250462] LustreError: 235608:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6817.286642] LustreError: 235608:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6821.946980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6822.378991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6826.755588] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6829.737407] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6836.144456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6836.615414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6843.914381] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6853.014395] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6854.246461] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6854.386454] Lustre: lustre-OST0001: new disk, initializing [ 6854.390515] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6854.402293] Lustre: 237916:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6854.517156] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6861.989818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6862.001792] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6862.147984] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6863.253651] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6876.171603] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6885.279952] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6898.155145] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6898.163655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6898.177597] Lustre: Skipped 3 previous similar messages [ 6898.191178] Lustre: Skipped 2 previous similar messages [ 6903.275801] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6903.607885] Lustre: server umount lustre-MDT0000 complete [ 6907.975766] LustreError: 235587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788933319 with bad export cookie 9109353265605672013 [ 6907.993955] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6907.994858] LustreError: 235587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6908.388701] LustreError: 238703:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6908.392439] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6908.415563] LustreError: 238703:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6908.440610] Lustre: Skipped 1 previous similar message [ 6908.728967] Lustre: server umount lustre-MDT0001 complete [ 6915.798577] Lustre: server umount lustre-OST0000 complete [ 6920.632727] Lustre: server umount lustre-OST0001 complete [ 6943.395378] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 6955.644798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6956.305712] LustreError: 241024:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6956.332458] LustreError: 241024:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6956.460757] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6961.414911] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6961.640300] LustreError: 241025:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6966.758185] LustreError: 241024:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6970.301110] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6970.876078] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6976.315866] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6979.522575] Lustre: 242166:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6986.761967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6987.286356] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6994.738386] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 6995.500805] LustreError: 242521:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7001.597993] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7003.278864] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7008.770217] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7009.694931] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7017.334693] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7020.617936] Lustre: 244036:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7032.293696] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7032.307975] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7032.327320] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7032.335922] Lustre: Skipped 3 previous similar messages [ 7034.337878] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7034.341877] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7034.346042] Lustre: Skipped 2 previous similar messages [ 7034.355578] Lustre: Skipped 2 previous similar messages [ 7036.194127] Lustre: server umount mylustre-MDT0000 complete [ 7039.470088] LustreError: 241025:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7039.495181] LustreError: 241025:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7039.835137] LustreError: 244235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788933451 with bad export cookie 9109353265605674820 [ 7039.842937] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7039.846471] LustreError: 244235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7040.051336] Lustre: server umount mylustre-MDT0001 complete [ 7055.149143] Lustre: server umount mylustre-OST0000 complete [ 7068.856676] Lustre: server umount mylustre-OST0001 complete [ 7089.980746] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 7100.680983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7101.223503] LustreError: 246346:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7101.307362] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7101.313290] Lustre: Skipped 1 previous similar message [ 7105.888951] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7113.684838] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7118.761462] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7122.234750] Lustre: 247488:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7132.122796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7140.575947] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7145.996230] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7150.303609] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7155.704350] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7156.896174] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7164.450235] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7168.074910] Lustre: 249360:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7175.650501] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7175.660570] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7175.685102] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7175.694355] Lustre: Skipped 1 previous similar message [ 7181.445212] Lustre: server umount tfs-MDT0000 complete [ 7185.173591] LustreError: 246323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788933596 with bad export cookie 9109353265605676703 [ 7185.190162] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7185.655782] Lustre: server umount tfs-MDT0001 complete [ 7200.201865] Lustre: server umount tfs-OST0000 complete [ 7202.272134] Lustre: 234873:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788933598/real 1788933598] req@ffff8f9ee44b6d80 x1875832423527936/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788933614 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7202.313533] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7202.336500] Lustre: Skipped 3 previous similar messages [ 7204.015226] Lustre: server umount tfs-OST0001 complete [ 7222.299824] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 7232.648741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7233.049550] LustreError: 251607:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7233.069428] LustreError: 251607:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 7233.110528] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7233.118240] Lustre: Skipped 3 previous similar messages [ 7238.548576] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7247.431639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7252.086032] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7255.399605] Lustre: 252761:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7261.043423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7267.225252] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7274.835020] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7279.232696] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7279.261451] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7281.025697] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7287.534658] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7290.931631] Lustre: 254631:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7297.161778] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:01:47 (1788933707) [ 7314.401927] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7314.412267] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7314.429164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7314.435586] Lustre: Skipped 8 previous similar messages [ 7318.961455] Lustre: server umount lustre-MDT0000 complete [ 7322.392552] LustreError: 251584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788933734 with bad export cookie 9109353265605678376 [ 7322.396686] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7322.404958] LustreError: 251584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7328.838763] Lustre: server umount lustre-MDT0001 complete [ 7338.424409] Lustre: server umount lustre-OST0000 complete [ 7356.384123] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7356.596424] Lustre: server umount lustre-OST0001 complete [ 7362.799103] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 7371.302661] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 7401.448622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7401.671094] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7401.718629] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7401.829568] Lustre: lustre-MDT0000: new disk, initializing [ 7401.931280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7401.935088] Lustre: Skipped 3 previous similar messages [ 7401.951252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7406.189611] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7415.931798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7416.079208] Lustre: 257815:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7416.147097] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7416.152438] Lustre: Skipped 1 previous similar message [ 7416.238890] Lustre: lustre-MDT0001: new disk, initializing [ 7416.383211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7416.398698] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7420.687535] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7431.487773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7431.789809] Lustre: lustre-OST0000: new disk, initializing [ 7431.794119] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7431.806876] Lustre: 259397:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7433.290070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7433.301477] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7433.464118] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7438.649147] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7448.743229] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7448.859813] Lustre: lustre-OST0001: new disk, initializing [ 7448.864942] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7448.878201] Lustre: 260269:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7450.641338] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7450.663612] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7450.782639] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7454.207857] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7465.727993] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:04:35 (1788933875) [ 7481.358247] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:04:50 (1788933890) [ 7500.257856] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7500.259857] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7500.261643] LustreError: Skipped 1 previous similar message [ 7500.264194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7500.277358] Lustre: Skipped 6 previous similar messages [ 7500.293796] Lustre: Skipped 9 previous similar messages [ 7503.195692] Lustre: server umount lustre-OST0000 complete [ 7506.917968] LustreError: 259383:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7506.958123] LustreError: 259383:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7521.249844] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7521.620100] Lustre: server umount lustre-MDT0000 complete [ 7525.360641] LustreError: 260268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788933937 with bad export cookie 9109353265605679930 [ 7525.367794] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7525.374212] LustreError: 260268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7531.880531] Lustre: server umount lustre-MDT0001 complete [ 7542.807699] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 7613.946993] Lustre: server umount lustre-OST0001 complete [ 7620.853662] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 7636.065889] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 7670.120482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7670.502402] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7670.536615] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7670.651167] Lustre: lustre-MDT0000: new disk, initializing [ 7670.742897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7670.748139] Lustre: Skipped 3 previous similar messages [ 7670.755237] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7675.499481] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7686.978622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7687.142448] Lustre: 265129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7687.188609] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7687.195620] Lustre: Skipped 1 previous similar message [ 7687.265899] Lustre: lustre-MDT0001: new disk, initializing [ 7687.398845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7687.409088] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7692.579665] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7702.004898] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7707.904669] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7715.119754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7715.526658] Lustre: 267098:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7717.468715] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7717.586952] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7721.872228] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7731.329523] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7738.338384] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7738.345026] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7738.351479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7738.370638] Lustre: Skipped 5 previous similar messages [ 7738.381805] Lustre: Skipped 13 previous similar messages [ 7744.166177] Lustre: server umount lustre-OST0000 complete [ 7753.806651] Lustre: server umount lustre-MDT0000 complete [ 7757.034416] LustreError: 265120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788934168 with bad export cookie 9109353265605681400 [ 7757.046678] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7757.331838] Lustre: server umount lustre-MDT0001 complete [ 7766.989798] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 7769.513350] Key type lgssc unregistered [ 7769.845644] LNet: 268860:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7769.853777] LNetError: 268860:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7769.870257] LNet: Removed LNI 192.168.206.155@tcp [ 7770.685165] Key type .llcrypt unregistered [ 7770.690548] Key type ._llcrypt unregistered [ 7795.917474] Key type ._llcrypt registered [ 7795.919665] Key type .llcrypt registered [ 7796.005160] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 7797.430189] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7797.498536] alg: No test for adler32 (adler32-zlib) [ 7798.616068] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 7798.946840] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 7800.706519] Key type lgssc registered [ 7802.009479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7809.376538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7809.389448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7810.950572] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7815.280422] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7820.261990] LustreError: 270325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7820.279956] LustreError: 270325:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7823.168710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7823.569870] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7827.700518] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7834.668942] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7840.343058] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7847.270133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7847.741290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7854.349906] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7855.934720] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7862.369634] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7863.294981] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7866.337425] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7866.344450] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7866.357628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7866.360373] Lustre: Skipped 1 previous similar message [ 7866.374620] Lustre: Skipped 1 previous similar message [ 7871.431454] Lustre: server umount lustre-OST0000 complete [ 7876.579717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7876.599395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7876.606601] Lustre: Skipped 1 previous similar message [ 7881.556305] Lustre: server umount lustre-MDT0000 complete [ 7881.698945] LustreError: 270321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7881.722713] LustreError: 270321:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7885.257646] LustreError: 270306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788934297 with bad export cookie 18405616335990647597 [ 7885.273912] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7885.274963] LustreError: 270306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7885.530320] Lustre: server umount lustre-MDT0001 complete [ 7896.648649] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 7899.581035] Key type lgssc unregistered [ 7899.917908] LNet: 273913:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7899.927440] LNetError: 273913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7899.946143] LNet: Removed LNI 192.168.206.155@tcp [ 7900.838872] Key type .llcrypt unregistered [ 7900.840752] Key type ._llcrypt unregistered [ 7918.040794] Key type ._llcrypt registered [ 7918.044236] Key type .llcrypt registered [ 7918.145502] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 7919.256280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7919.287335] alg: No test for adler32 (adler32-zlib) [ 7920.298121] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 7920.458278] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 7922.144150] Key type lgssc registered [ 7923.443960] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7933.566841] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7933.597681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7949.152666] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7954.273361] LustreError: 275109:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.155@tcp: failed processing log, type 4: rc = -110 [ 7981.024175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7987.221459] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 7995.166215] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7997.123301] Lustre: server umount lustre-OST0000 complete [ 8013.728192] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 8015.747619] Key type lgssc unregistered [ 8015.955938] LNet: 276745:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8015.963943] LNetError: 276745:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8015.976906] LNet: Removed LNI 192.168.206.155@tcp [ 8016.627370] Key type .llcrypt unregistered [ 8016.629316] Key type ._llcrypt unregistered [ 8053.569463] Key type ._llcrypt registered [ 8053.571952] Key type .llcrypt registered [ 8053.705980] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 8054.842051] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8054.875739] alg: No test for adler32 (adler32-zlib) [ 8056.038836] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 8056.271067] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 8057.952220] Key type lgssc registered [ 8059.307150] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8073.288672] loop0: detected capacity change from 0 to 204800000 [ 8073.497740] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8073.513335] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8073.873861] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8074.882731] loop0: detected capacity change from 0 to 204800000 [ 8075.040938] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8075.057939] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8075.508647] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8076.617577] loop0: detected capacity change from 0 to 204800000 [ 8076.877277] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8076.886590] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8077.104424] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8078.191400] loop0: detected capacity change from 0 to 204800000 [ 8078.415705] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8078.552548] print_req_error: 1 callbacks suppressed [ 8078.552560] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8079.614551] loop0: detected capacity change from 0 to 204800000 [ 8079.641461] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8083.904185] loop0: detected capacity change from 0 to 204800000 [ 8083.937527] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8088.170275] loop0: detected capacity change from 0 to 204800000 [ 8088.221995] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8092.425516] loop0: detected capacity change from 0 to 204800000 [ 8092.454654] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8096.461381] loop0: detected capacity change from 0 to 204800000 [ 8096.567517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8096.629892] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8096.724104] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8096.740711] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8096.751081] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8096.762799] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8096.774302] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8096.800300] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8096.815622] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8096.832486] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8096.845332] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8096.856513] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8096.867960] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8096.893600] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8096.918313] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8096.931440] Lustre: 280157:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8102.440123] Lustre: server umount MGS complete [ 8103.208188] loop0: detected capacity change from 0 to 204800000 [ 8103.315267] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8103.804222] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8103.849515] Lustre: lustre-MDT0000: Aborting client recovery [ 8104.870827] loop1: detected capacity change from 0 to 204800000 [ 8104.967864] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8105.058171] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8105.078666] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8105.100539] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8105.123253] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8105.141671] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8105.154162] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8105.167473] Lustre: 280591:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8105.197375] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8105.350312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8105.418627] Lustre: lustre-MDT0001: Aborting client recovery [ 8106.382831] loop2: detected capacity change from 0 to 204800000 [ 8106.590098] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8106.709843] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8106.721267] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8106.741840] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8106.768797] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8106.777355] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8106.786146] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8106.794347] Lustre: 280669:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8106.963662] Lustre: Found index 0 for lustre-OST0000, updating log [ 8106.991630] Lustre: 280687:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8107.049991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8107.063790] Lustre: lustre-OST0000: Aborting recovery [ 8107.961385] loop3: detected capacity change from 0 to 204800000 [ 8108.131620] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8108.216885] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8108.236366] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8108.255535] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8108.265093] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8108.274706] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8108.288810] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8108.300656] Lustre: 280752:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8108.322757] Lustre: Found index 1 for lustre-OST0001, updating log [ 8108.346523] Lustre: 280757:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8108.418453] Lustre: lustre-OST0001: Aborting recovery [ 8113.512275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8113.513545] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8113.542548] LustreError: 280535:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8113.561694] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8113.582070] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8115.561708] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8115.567947] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8115.852738] LustreError: 280502:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 489us: rc = -2 [ 8116.382097] LustreError: 280694:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1190us: rc = -2 [ 8116.402274] LustreError: 280694:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 28 previous similar messages [ 8131.041644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8131.042679] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8131.046504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8131.076497] Lustre: Skipped 3 previous similar messages [ 8135.018501] Lustre: server umount lustre-MDT0000 complete [ 8135.810495] LustreError: 280484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788934547 with bad export cookie 996967879633493524 [ 8135.821096] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8136.161725] LustreError: 280501:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8136.161906] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8136.169641] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8136.169651] Lustre: Skipped 3 previous similar messages [ 8136.173198] LustreError: 280501:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8136.210972] Lustre: Skipped 1 previous similar message [ 8140.193056] LustreError: 280694:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8140.194251] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8140.220433] Lustre: Skipped 2 previous similar messages [ 8142.337887] Lustre: server umount lustre-MDT0001 complete [ 8149.460121] Lustre: server umount lustre-OST0000 complete [ 8156.610927] Lustre: server umount lustre-OST0001 complete [ 8164.234381] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:16:14 (1788934574) [ 8192.071679] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 8204.319171] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 8236.258135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8236.531880] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8236.576682] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8236.698316] Lustre: lustre-MDT0000: new disk, initializing [ 8236.813384] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8236.823412] Lustre: Skipped 1 previous similar message [ 8236.844951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8240.588913] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8250.271203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8250.428572] Lustre: 284822:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8250.474995] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8250.488599] Lustre: Skipped 1 previous similar message [ 8250.591240] Lustre: lustre-MDT0001: new disk, initializing [ 8250.715850] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8250.750540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8250.770147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8255.601444] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8264.898892] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8269.476297] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8275.397261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8275.603414] Lustre: lustre-OST0000: new disk, initializing [ 8275.608915] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8275.615430] Lustre: 286789:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8275.697756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8282.115213] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8285.739852] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8285.749859] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8285.869338] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8291.442586] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8296.277683] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8306.892472] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8315.573838] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8324.372211] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8373.218865] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8373.231973] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8373.235035] LustreError: Skipped 1 previous similar message [ 8373.246776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8374.686573] Lustre: server umount lustre-OST0000 complete [ 8378.337795] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8378.368671] Lustre: Skipped 1 previous similar message [ 8378.374675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8378.380081] Lustre: Skipped 2 previous similar messages [ 8384.526696] Lustre: server umount lustre-MDT0000 complete [ 8388.172985] LustreError: 284814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788934799 with bad export cookie 996967879633496233 [ 8388.184419] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8388.184746] LustreError: 284814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8388.513821] Lustre: server umount lustre-MDT0001 complete [ 8395.581607] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8400.367091] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8405.361497] Lustre: server umount MGS complete [ 8417.121713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8417.600777] LustreError: 290760:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8417.626372] LustreError: 290760:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8417.654938] LustreError: 290773:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8417.766438] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8422.324425] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8422.889438] LustreError: 290761:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8428.007451] LustreError: 290760:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8431.091410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8435.998626] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8443.378242] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8448.959628] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8455.306792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8455.647845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8455.662843] Lustre: Skipped 1 previous similar message [ 8461.359867] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8467.854167] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8496.371649] Lustre: server umount lustre-OST0000 complete [ 8503.269528] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8503.277634] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8503.290361] Lustre: Skipped 1 previous similar message [ 8503.294901] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8503.303637] Lustre: Skipped 2 previous similar messages [ 8509.989108] Lustre: server umount lustre-MDT0000 complete [ 8513.749393] LustreError: 290741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788934925 with bad export cookie 996967879633498186 [ 8513.755679] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8513.764989] LustreError: 290741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8523.422133] Lustre: server umount lustre-MDT0001 complete [ 8534.535253] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 8537.383419] Key type lgssc unregistered [ 8537.660705] LNet: 294708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8537.670332] LNetError: 294708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8537.691480] LNet: Removed LNI 192.168.206.155@tcp [ 8538.466744] Key type .llcrypt unregistered [ 8538.469319] Key type ._llcrypt unregistered [ 8574.619115] Key type ._llcrypt registered [ 8574.621719] Key type .llcrypt registered [ 8574.762115] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 8588.782656] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 8589.433321] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8589.565948] alg: No test for adler32 (adler32-zlib) [ 8590.717095] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 8590.940792] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 8592.592133] Key type lgssc registered [ 8593.839982] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8625.018879] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8625.032792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8626.243627] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8626.268154] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8626.347887] Lustre: lustre-MDT0000: new disk, initializing [ 8626.409837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8626.423901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8630.138095] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8639.459825] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8639.543128] Lustre: 298663:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8639.611597] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8639.618547] Lustre: Skipped 1 previous similar message [ 8639.694465] Lustre: lustre-MDT0001: new disk, initializing [ 8639.811404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8639.866515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8639.902172] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8643.245251] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8651.340709] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8656.741442] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8663.266947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8663.580910] Lustre: lustre-OST0000: new disk, initializing [ 8663.588833] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8663.603126] Lustre: 300633:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8663.715881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8669.708248] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8671.319842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8671.337971] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8671.510512] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8679.391177] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8686.970205] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8700.055520] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8703.961769] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8713.818201] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8753.638705] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8753.645728] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8753.651148] LustreError: Skipped 1 previous similar message [ 8753.656588] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8753.680419] Lustre: Skipped 1 previous similar message [ 8757.963862] Lustre: server umount lustre-OST0000 complete [ 8762.849580] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8762.859969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8762.878660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8762.882770] Lustre: Skipped 1 previous similar message [ 8763.874263] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8763.885903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8763.889860] Lustre: Skipped 1 previous similar message [ 8769.372652] Lustre: server umount lustre-MDT0000 complete [ 8772.516624] LustreError: 298653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788935184 with bad export cookie 17119224240972942909 [ 8772.520863] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8772.531498] LustreError: 298653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8772.834706] Lustre: server umount lustre-MDT0001 complete [ 8780.816195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8785.516897] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8790.209770] Lustre: server umount MGS complete [ 8800.591440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8800.984940] LustreError: 304503:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8801.002143] LustreError: 304503:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8801.014981] LustreError: 304516:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8801.108624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8804.850783] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8806.370593] LustreError: 304504:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8806.387164] LustreError: 304504:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8810.469934] LustreError: 304503:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8812.084459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8812.333809] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8816.174555] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8822.383830] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8827.554096] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8833.134696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8833.422046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8839.429660] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8845.869089] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8876.516364] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8876.519268] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8876.526706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8876.556959] Lustre: Skipped 1 previous similar message [ 8881.636699] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8881.648369] Lustre: Skipped 2 previous similar messages [ 8882.371269] Lustre: server umount lustre-OST0000 complete [ 8886.754290] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8886.765492] Lustre: Skipped 1 previous similar message [ 8891.278622] Lustre: server umount lustre-MDT0000 complete [ 8891.879728] LustreError: 304500:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8891.899456] LustreError: 304500:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8893.992427] LustreError: 305240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788935305 with bad export cookie 17119224240972944841 [ 8893.995858] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8894.002539] LustreError: 305240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8894.237438] Lustre: server umount lustre-MDT0001 complete [ 8903.542915] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [ 8906.042196] Key type lgssc unregistered [ 8906.254867] LNet: 308450:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8906.262387] LNetError: 308450:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8906.271762] LNet: Removed LNI 192.168.206.155@tcp [ 8906.970683] Key type .llcrypt unregistered [ 8906.973409] Key type ._llcrypt unregistered [ 8926.400959] Key type ._llcrypt registered [ 8926.405477] Key type .llcrypt registered [ 8926.512715] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 8938.551132] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 8939.190801] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8939.215637] alg: No test for adler32 (adler32-zlib) [ 8940.290074] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [ 8940.582327] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [ 8942.240147] Key type lgssc registered [ 8943.179542] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8963.259803] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 8976.073839] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8976.098589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8977.345534] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8977.377259] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8977.435696] Lustre: lustre-MDT0000: new disk, initializing [ 8977.487520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8977.504161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8981.528756] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 8991.251207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8991.528314] Lustre: lustre-OST0000: new disk, initializing [ 8991.533692] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8991.538760] Lustre: Skipped 1 previous similar message [ 8991.547293] Lustre: 312481:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8991.657629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8993.365516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8993.376679] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8993.415033] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8997.541192] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9748.047928] Lustre: server umount lustre-MDT0000 complete [ 9751.501573] LustreError: 311665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788936163 with bad export cookie 10377780537668889278 [ 9751.525494] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9761.931434] Lustre: server umount lustre-OST0000 complete [ 9786.454436] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [ 9795.788223] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [ 9828.202276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9828.557108] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9828.600602] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9828.703305] Lustre: lustre-MDT0000: new disk, initializing [ 9828.803171] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9828.822990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9833.708944] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9844.701020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9844.947763] Lustre: 316799:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9845.009402] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9845.013151] Lustre: Skipped 1 previous similar message [ 9845.162450] Lustre: lustre-MDT0001: new disk, initializing [ 9845.254789] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9845.280224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9845.291531] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9848.319404] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9856.737894] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9861.102501] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9867.083226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9867.375266] Lustre: lustre-OST0000: new disk, initializing [ 9867.379778] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9867.386743] Lustre: 318769:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9867.482204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9873.864481] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9877.078354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9877.094574] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9877.257991] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9883.649251] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9888.245440] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9888.415978] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9893.303105] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9893.496636] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9897.954062] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9897.955671] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9897.965691] LustreError: Skipped 1 previous similar message [ 9897.967978] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9898.003101] Lustre: Skipped 1 previous similar message [ 9901.416484] Lustre: server umount lustre-OST0000 complete [ 9906.659786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9906.665868] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9906.686375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9906.697103] Lustre: Skipped 1 previous similar message [ 9908.194702] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9908.199064] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9911.761301] Lustre: server umount lustre-MDT0000 complete [ 9915.606829] LustreError: 316791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788936327 with bad export cookie 10377780537671830097 [ 9915.620883] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9915.634207] LustreError: 316791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9916.376158] Lustre: server umount lustre-MDT0001 complete [ 9920.474770] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9922.352856] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 02:45:31 (1788936331) [ 9929.757066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9930.177413] LustreError: 321083:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9930.198618] LustreError: 321083:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9930.281337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9934.765381] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9935.333568] LustreError: 321084:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9939.436149] LustreError: 321083:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9942.451248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9942.926643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9948.016981] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9956.174795] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9962.057677] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9969.029706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9969.383140] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9974.997184] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9981.432951] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9987.514519] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9987.643429] Lustre: lustre-OST0001: new disk, initializing [ 9987.646436] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9987.651219] Lustre: 323921:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9987.729655] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9993.311573] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [ 9993.839045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9993.850415] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9993.916935] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9993.933809] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9994.983407] LustreError: 324303:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9994.998621] LustreError: 324303:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9996.010632] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9998.055304] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9998.067160] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10002.210377] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10002.223019] LustreError: 324298:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10002.570260] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10002.800191] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10007.965235] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c9e48581800.ost_server_uuid 50 [10009.227761] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c9e48581800.ost_server_uuid in FULL state after 0 sec [10010.411829] LustreError: 324303:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10010.424798] LustreError: 324303:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10014.313449] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c9e48581800.ost_server_uuid 50 [10015.643511] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c9e48581800.ost_server_uuid in FULL state after 0 sec [10017.654675] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10033.122374] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10033.123186] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10033.136160] LustreError: Skipped 1 previous similar message [10033.139825] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10033.158558] Lustre: Skipped 1 previous similar message [10033.177991] Lustre: Skipped 1 previous similar message [10036.724249] Lustre: server umount lustre-OST0001 complete [10039.543795] LustreError: 324305:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10045.420933] LustreError: 322907:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10045.427693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10045.458332] LustreError: 322907:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10045.464467] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10045.464475] Lustre: Skipped 4 previous similar messages [10045.495769] Lustre: Skipped 1 previous similar message [10045.499283] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10048.520540] Lustre: server umount lustre-OST0000 complete [10055.652067] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10055.654805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10055.661571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10055.661579] Lustre: Skipped 1 previous similar message [10055.666432] Lustre: Skipped 1 previous similar message [10066.404792] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10066.595663] Lustre: server umount lustre-MDT0000 complete [10070.174656] LustreError: 321063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788936481 with bad export cookie 10377780537671830979 [10070.176126] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10070.182691] LustreError: 321063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10070.549687] Lustre: server umount lustre-MDT0001 complete [10079.976992] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [10082.815964] Key type lgssc unregistered [10083.160565] LNet: 326609:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10083.166779] LNetError: 326609:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10083.178605] LNet: Removed LNI 192.168.206.155@tcp [10083.938450] Key type .llcrypt unregistered [10083.944221] Key type ._llcrypt unregistered [10107.166485] Key type ._llcrypt registered [10107.169128] Key type .llcrypt registered [10107.327702] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [10108.474947] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10108.766710] alg: No test for adler32 (adler32-zlib) [10109.852801] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [10110.117896] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [10111.832582] Key type lgssc registered [10112.985836] Lustre: Echo OBD driver; http://www.lustre.org/ [10120.593908] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10120.613690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10122.157593] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10126.217931] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10131.435814] LustreError: 328031:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10131.457091] LustreError: 328031:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10133.532213] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10136.550081] LustreError: 328032:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10139.829502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10140.170986] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10143.877365] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10151.662846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10151.975734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10156.903460] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10163.360708] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10164.269869] LustreError: 329712:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10166.318994] LustreError: 329712:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10166.334295] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10166.342031] LustreError: 329712:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10168.957905] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10169.248087] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10174.462106] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10175.910384] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10183.480627] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10189.776479] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10190.069651] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10292.193811] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10292.194264] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10292.200740] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10292.200774] LustreError: Skipped 1 previous similar message [10292.225647] Lustre: Skipped 1 previous similar message [10295.323972] Lustre: server umount lustre-OST0001 complete [10297.330736] LustreError: 329710:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10297.341976] LustreError: 329710:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10302.438566] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10302.439658] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10302.465136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10302.495074] Lustre: Skipped 2 previous similar messages [10307.519342] Lustre: server umount lustre-OST0000 complete [10312.677763] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10312.699357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10312.716990] Lustre: Skipped 2 previous similar messages [10317.797853] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10317.819412] Lustre: Skipped 2 previous similar messages [10318.179421] Lustre: server umount lustre-MDT0000 complete [10322.603430] LustreError: 328013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788936734 with bad export cookie 4667802205101971966 [10322.608392] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10322.621374] LustreError: 328013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10323.066373] Lustre: server umount lustre-MDT0001 complete [10332.511969] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [10334.812981] Key type lgssc unregistered [10335.056472] LNet: 332974:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10335.060659] LNetError: 332974:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10336.104521] LNet: Removed LNI 192.168.206.155@tcp [10336.825853] Key type .llcrypt unregistered [10336.827706] Key type ._llcrypt unregistered [10380.667481] Key type ._llcrypt registered [10380.680791] Key type .llcrypt registered [10380.818711] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [10381.986684] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10382.142128] alg: No test for adler32 (adler32-zlib) [10383.211675] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [10383.428721] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [10385.080452] Key type lgssc registered [10386.306284] Lustre: Echo OBD driver; http://www.lustre.org/ [10395.336526] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10395.384080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10396.925169] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10400.934708] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10406.376301] LustreError: 335264:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10406.410722] LustreError: 335264:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10409.715736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10410.176240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10414.688778] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10417.525590] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10424.328457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10424.610383] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10431.093725] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10434.869300] LustreError: 336756:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10438.804398] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10439.034229] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10442.115203] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10442.188058] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10444.983633] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10452.558615] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10460.118113] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10472.951791] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10472.961084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10472.990539] Lustre: Skipped 2 previous similar messages [10472.994239] Lustre: Skipped 1 previous similar message [10478.055926] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10478.070113] Lustre: Skipped 5 previous similar messages [10478.881556] Lustre: server umount lustre-MDT0000 complete [10482.736318] LustreError: 335291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788936894 with bad export cookie 13481292525012134780 [10482.751652] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10483.175228] LustreError: 338615:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10483.183345] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10483.207475] Lustre: Skipped 2 previous similar messages [10483.360248] Lustre: server umount lustre-MDT0001 complete [10489.213652] Lustre: server umount lustre-OST0000 complete [10492.537776] Lustre: server umount lustre-OST0001 complete [10509.003732] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [10520.400443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10520.929472] LustreError: 340527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10520.947710] LustreError: 340527:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10521.013845] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10525.001292] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10526.178854] LustreError: 340528:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10533.872985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10534.309470] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10538.776666] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10541.440511] Lustre: 341672:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10547.752365] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10554.072589] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10558.461281] LustreError: 342026:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10558.486921] LustreError: 342026:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10563.539128] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10563.882803] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10563.889630] Lustre: Skipped 1 previous similar message [10565.956509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10565.958408] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10571.008112] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10580.158379] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10584.472199] Lustre: 343542:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10590.691091] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10590.707598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10590.724029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10595.166233] Lustre: server umount lustre-MDT0000 complete [10596.845431] LustreError: 340528:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10596.866710] LustreError: 340528:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10598.504374] LustreError: 340507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788937010 with bad export cookie 13481292525012136481 [10598.506937] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10598.515893] LustreError: 340507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [10599.023104] Lustre: server umount lustre-MDT0001 complete [10612.677133] Lustre: server umount lustre-OST0000 complete [10626.891867] Lustre: server umount lustre-OST0001 complete [10629.476819] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10630.910361] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10632.265626] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 02:57:22 (1788937042) [10654.179047] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [10662.119824] loop0: detected capacity change from 0 to 1099511627776 [10662.652027] SGI XFS with ACLs, security attributes, quota, no debug enabled [10662.662580] *** VALIDATE xfs *** [10662.679557] XFS (loop0): Mounting V5 Filesystem [10662.762806] XFS (loop0): Ending clean mount [10663.932250] loop1: detected capacity change from 0 to 18691697672192 [10756.769251] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [10767.435603] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [10801.617807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10801.825956] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10801.853155] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10801.923992] Lustre: lustre-MDT0000: new disk, initializing [10802.000810] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10802.019469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10806.505915] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10816.362871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10816.478741] Lustre: 348675:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10816.547658] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10816.557505] Lustre: Skipped 1 previous similar message [10816.648652] Lustre: lustre-MDT0001: new disk, initializing [10816.769435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10816.791652] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10819.784177] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10827.804619] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10832.695775] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10838.673831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10838.978533] Lustre: lustre-OST0000: new disk, initializing [10838.986608] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10838.991461] Lustre: 350641:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10840.220960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10840.227638] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10840.341143] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10844.207385] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10852.810725] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10857.475887] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10857.708383] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10861.774765] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10861.942833] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10866.151146] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10866.151843] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10866.158281] LustreError: Skipped 1 previous similar message [10866.165497] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10866.178018] Lustre: Skipped 4 previous similar messages [10866.189836] Lustre: Skipped 5 previous similar messages [10869.552747] Lustre: server umount lustre-OST0000 complete [10873.313411] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10873.319069] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10873.332352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10879.152626] Lustre: server umount lustre-MDT0000 complete [10881.505059] LustreError: 348687:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10881.519521] LustreError: 348687:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10882.857577] LustreError: 348666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788937294 with bad export cookie 13481292525012138035 [10882.866276] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10882.872855] LustreError: 348666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10883.226501] Lustre: server umount lustre-MDT0001 complete [10891.666734] XFS (loop0): Unmounting Filesystem [10895.526698] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:01:45 (1788937305) [10901.610158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10902.064478] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10902.074469] Lustre: Skipped 2 previous similar messages [10907.343370] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10916.961018] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10922.588254] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10929.708967] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10933.825313] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10939.418624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10946.232372] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [10954.121574] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10963.430179] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10963.453653] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10963.458160] LustreError: Skipped 1 previous similar message [10963.463678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10963.470695] Lustre: Skipped 2 previous similar messages [10963.487516] Lustre: Skipped 3 previous similar messages [10969.044460] Lustre: server umount lustre-OST0000 complete [10978.727488] Lustre: server umount lustre-MDT0000 complete [10978.806655] LustreError: 353207:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10978.836890] LustreError: 353207:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [10983.192025] LustreError: 353192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788937394 with bad export cookie 13481292525012138910 [10983.198363] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10983.207642] LustreError: 353192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10983.579352] Lustre: server umount lustre-MDT0001 complete [10998.161766] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [11001.232449] Key type lgssc unregistered [11001.504919] LNet: 356844:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11001.521511] LNetError: 356844:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11001.561976] LNet: Removed LNI 192.168.206.155@tcp [11002.663142] Key type .llcrypt unregistered [11002.671580] Key type ._llcrypt unregistered [11036.516225] Key type ._llcrypt registered [11036.517967] Key type .llcrypt registered [11036.657124] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [11038.255581] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11038.304057] alg: No test for adler32 (adler32-zlib) [11039.358819] Lustre: Lustre: Build Version: 2.17.57_55_ga2ebe9c [11039.631239] LNet: Added LNI 192.168.206.155@tcp [8/256/0/180] [11041.352202] Key type lgssc registered [11042.412606] Lustre: Echo OBD driver; http://www.lustre.org/ [11049.325567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11049.379326] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11050.747710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11055.132876] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [11061.222383] LustreError: 358412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11061.247977] LustreError: 358412:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11064.756804] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11065.326524] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11069.054630] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [11075.731127] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11081.463754] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11087.710529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11088.013841] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11095.069275] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [11102.783568] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11108.949196] Lustre: server umount lustre-MDT0001 complete [11109.625664] LustreError: 358408:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11111.394417] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11111.400528] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11111.421368] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11111.913359] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11114.728869] LustreError: 358409:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11114.758965] LustreError: 358409:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11117.862422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11117.942969] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11117.952601] Lustre: Found index 1 for lustre-MDT0001, updating log [11118.009318] Lustre: 358402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11118.381167] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11122.879520] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [11123.698336] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11127.788554] LustreError: 362013:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11127.794751] LustreError: 362013:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11127.810096] LustreError: 358404:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11127.854927] LustreError: 362016:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.155@tcp: cfg command failed: rc = -17 [11127.862397] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11127.862397] [11428.834608] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11428.853147] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11428.858643] Lustre: Skipped 1 previous similar message [11432.199711] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.55@tcp (stopping) [11433.960060] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11437.298210] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.55@tcp (stopping) [11437.311044] Lustre: Skipped 1 previous similar message [11442.145334] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11442.413545] LustreError: 358408:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11442.441681] LustreError: 358408:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11442.596148] Lustre: server umount lustre-MDT0001 complete [11452.658899] LustreError: 358408:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11452.679206] LustreError: 358408:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11453.562161] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11453.720377] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11453.740113] Lustre: Found index 1 for lustre-MDT0001, updating log [11453.802664] Lustre: 358401:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11454.184307] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11458.904588] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [11459.567495] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11462.661231] LustreError: 370737:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11462.671299] LustreError: 370737:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11462.679110] LustreError: 358404:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11462.688369] LustreError: 358404:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11462.729679] LustreError: 370740:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11462.738864] LustreError: 370740:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11462.743407] LustreError: 370740:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.155@tcp: cfg command failed: rc = -17 [11462.753880] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.155@tcp [11462.753880] [11764.988450] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.55@tcp (stopping) [11765.000327] Lustre: Skipped 2 previous similar messages [11766.759242] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11766.785431] Lustre: Skipped 1 previous similar message [11766.793064] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11770.564996] Lustre: server umount lustre-MDT0001 complete [11771.875948] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11771.899024] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11781.455849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11781.577280] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11781.584819] Lustre: Found index 1 for lustre-MDT0001, updating log [11781.639207] Lustre: 358402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11781.904420] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11787.082846] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [11787.253772] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11790.847634] LustreError: 379411:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11790.853908] LustreError: 379411:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11790.859304] LustreError: 358404:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11790.863296] LustreError: 358404:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11790.929605] LustreError: 379414:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11790.948311] LustreError: 379414:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.155@tcp: cfg command failed: rc = -114 [11790.957823] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11790.957823] [12109.794268] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12109.800080] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12109.807610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12109.807619] Lustre: Skipped 3 previous similar messages [12109.856837] Lustre: Skipped 3 previous similar messages [12115.696901] Lustre: server umount lustre-MDT0000 complete [12119.620051] LustreError: 358392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788938531 with bad export cookie 930007815142877725 [12119.627878] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12119.638821] LustreError: 358392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12120.034837] LustreError: 358407:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12120.034981] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12120.066409] LustreError: 358407:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12126.255586] Lustre: server umount lustre-MDT0001 complete [12130.486271] Lustre: server umount lustre-OST0000 complete [12139.813718] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [12150.993884] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [12189.006574] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:23:18 (1788938598) [12196.001335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12196.330762] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12196.389253] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12196.511605] Lustre: lustre-MDT0000: new disk, initializing [12196.592937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12196.614072] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12201.078668] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [12211.456532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12211.589712] Lustre: 390344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12211.667483] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12211.672502] Lustre: Skipped 1 previous similar message [12211.747778] Lustre: lustre-MDT0001: new disk, initializing [12211.843927] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12211.877344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12211.891120] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12216.512792] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [12225.835776] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12232.075339] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12239.867146] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12240.191430] Lustre: lustre-OST0000: new disk, initializing [12240.196340] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12240.205319] Lustre: 392313:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12240.327859] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12246.127028] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12246.154152] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12246.288951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12247.506343] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_default_debug -1 all [12257.633363] Lustre: DEBUG MARKER: oleg655-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12266.978633] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12266.981907] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12267.006917] Lustre: Skipped 2 previous similar messages [12267.035975] Lustre: Skipped 9 previous similar messages [12267.555205] Lustre: server umount lustre-MDT0000 complete [12271.396795] LustreError: 391140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788938683 with bad export cookie 930007815142879678 [12271.409215] LustreError: 391140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12271.409765] LustreError: MGC192.168.206.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12271.880477] Lustre: server umount lustre-MDT0001 complete [12286.252478] Lustre: server umount lustre-OST0000 complete [12318.272545] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing set_hostid [12329.808773] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing load_modules_local [12361.955832] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12060 sec ========= 03:26:11 (1788938771) [12363.719949] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:26:13 (1788938773) === [12367.203047] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:26:16 (1788938776) === [12408.675805] Lustre: DEBUG MARKER: oleg655-server.virtnet: executing unload_modules_local [12411.759896] Key type lgssc unregistered [12412.040698] LNet: 398682:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12412.051154] LNetError: 398682:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12412.094145] LNet: Removed LNI 192.168.206.155@tcp [12413.000380] Key type .llcrypt unregistered [12413.006249] Key type ._llcrypt unregistered