[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 478948884 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003272] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008585] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010013] pid_max: default: 32768 minimum: 301 [ 0.011107] LSM: Security Framework initializing [ 0.012064] Yama: becoming mindful. [ 0.013057] SELinux: Initializing. [ 0.014057] *** VALIDATE selinux *** [ 0.022625] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026834] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028119] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029105] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030114] *** VALIDATE tmpfs *** [ 0.031469] *** VALIDATE proc *** [ 0.032254] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.035182] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036116] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038029] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.041732] debug: unmapping init [mem 0xffffffff8ee59000-0xffffffff8ee60fff] [ 0.043182] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044716] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045020] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048016] ... value mask: 0000ffffffffffff [ 0.049016] ... max period: 00007fffffffffff [ 0.050017] ... fixed-purpose events: 3 [ 0.051014] ... event mask: 000000070000000f [ 0.053227] rcu: Hierarchical SRCU implementation. [ 0.055439] smp: Bringing up secondary CPUs ... [ 0.056638] x86: Booting SMP configuration: [ 0.057027] .... node #0, CPUs: #1 #2 #3 [ 0.061121] smp: Brought up 1 node, 4 CPUs [ 0.063018] smpboot: Max logical packages: 1 [ 0.064015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.207352] node 0 deferred pages initialised in 141ms [ 0.212036] devtmpfs: initialized [ 0.213396] x86/mm: Memory block size: 128MB [ 0.215784] gcov: version magic: 0x41383552 [ 0.217280] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.218070] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.219310] pinctrl core: initialized pinctrl subsystem [ 0.220270] [ 0.220859] ************************************************************* [ 0.221018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.222020] ** ** [ 0.223019] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.224019] ** ** [ 0.225016] ** This means that this kernel is built to expose internal ** [ 0.226021] ** IOMMU data structures, which may compromise security on ** [ 0.227023] ** your system. ** [ 0.228017] ** ** [ 0.229017] ** If you see this message and you are not debugging the ** [ 0.230018] ** kernel, report this immediately to your vendor! ** [ 0.231013] ** ** [ 0.232017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.233011] ************************************************************* [ 0.234731] NET: Registered protocol family 16 [ 0.235456] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.236058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.237064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.238628] cpuidle: using governor menu [ 0.239828] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.243645] PCI: Using configuration type 1 for base access [ 0.246123] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.256120] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.257023] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.259040] cryptd: max_cpu_qlen set to 1000 [ 0.262443] ACPI: Added _OSI(Module Device) [ 0.264034] ACPI: Added _OSI(Processor Device) [ 0.265021] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.267023] ACPI: Added _OSI(Processor Aggregator Device) [ 0.272907] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.279566] ACPI: Interpreter enabled [ 0.280089] ACPI: PM: (supports S0 S3 S4 S5) [ 0.281015] ACPI: Using IOAPIC for interrupt routing [ 0.282106] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.283462] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.292173] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.294086] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.297030] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.301142] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.307419] acpiphp: Slot [2] registered [ 0.309180] acpiphp: Slot [5] registered [ 0.311251] acpiphp: Slot [6] registered [ 0.313146] acpiphp: Slot [7] registered [ 0.314116] acpiphp: Slot [8] registered [ 0.316109] acpiphp: Slot [9] registered [ 0.318112] acpiphp: Slot [10] registered [ 0.319109] acpiphp: Slot [3] registered [ 0.321094] acpiphp: Slot [4] registered [ 0.323135] acpiphp: Slot [11] registered [ 0.324106] acpiphp: Slot [12] registered [ 0.326095] acpiphp: Slot [13] registered [ 0.328134] acpiphp: Slot [14] registered [ 0.329135] acpiphp: Slot [15] registered [ 0.331139] acpiphp: Slot [16] registered [ 0.333154] acpiphp: Slot [17] registered [ 0.334134] acpiphp: Slot [18] registered [ 0.335123] acpiphp: Slot [19] registered [ 0.337127] acpiphp: Slot [20] registered [ 0.338205] acpiphp: Slot [21] registered [ 0.340210] acpiphp: Slot [22] registered [ 0.341120] acpiphp: Slot [23] registered [ 0.343124] acpiphp: Slot [24] registered [ 0.344128] acpiphp: Slot [25] registered [ 0.345107] acpiphp: Slot [26] registered [ 0.347122] acpiphp: Slot [27] registered [ 0.348112] acpiphp: Slot [28] registered [ 0.349204] acpiphp: Slot [29] registered [ 0.351115] acpiphp: Slot [30] registered [ 0.352171] acpiphp: Slot [31] registered [ 0.354088] PCI host bridge to bus 0000:00 [ 0.355021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.357037] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.359029] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.362032] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.364029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.367036] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.368245] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.372180] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.375403] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.386864] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.392114] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.395023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.398021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.401087] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.404670] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.407868] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.412053] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.415688] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.420018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.433822] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.438020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.443226] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.454018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.462018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.477016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.486776] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.495023] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.507025] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.529023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.540230] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.547025] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.554026] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.567028] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.578000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.583015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.589017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.620020] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.630832] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.638022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.647027] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.665022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.677121] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.685024] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.694023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.712017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.723000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.725462] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.728459] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.731473] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.734337] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.738241] iommu: Default domain type: Passthrough [ 0.740496] SCSI subsystem initialized [ 0.742149] ACPI: bus type USB registered [ 0.744108] usbcore: registered new interface driver usbfs [ 0.746148] usbcore: registered new interface driver hub [ 0.748127] usbcore: registered new device driver usb [ 0.750236] pps_core: LinuxPPS API ver. 1 registered [ 0.752017] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.756094] PTP clock support registered [ 0.760015] EDAC MC: Ver: 3.0.0 [ 0.762186] PCI: Using ACPI for IRQ routing [ 0.765093] NetLabel: Initializing [ 0.766014] NetLabel: domain hash size = 128 [ 0.768015] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.770104] NetLabel: unlabeled traffic allowed by default [ 0.772201] vgaarb: loaded [ 0.773301] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.775024] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.782342] clocksource: Switched to clocksource kvm-clock [ 0.889508] VFS: Disk quotas dquot_6.6.0 [ 0.892668] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.895776] *** VALIDATE ramfs *** [ 0.897344] *** VALIDATE hugetlbfs *** [ 0.899137] pnp: PnP ACPI init [ 0.902334] pnp: PnP ACPI: found 6 devices [ 0.932828] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.936446] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.939027] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.941399] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.944075] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.946761] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.949047] NET: Registered protocol family 2 [ 0.951757] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.957096] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.974540] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.981742] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.986099] TCP: Hash tables configured (established 65536 bind 65536) [ 0.989870] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.993636] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.997023] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.000278] NET: Registered protocol family 1 [ 1.004897] RPC: Registered named UNIX socket transport module. [ 1.007535] RPC: Registered udp transport module. [ 1.009285] RPC: Registered tcp transport module. [ 1.011163] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.014281] NET: Registered protocol family 44 [ 1.016340] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.018742] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.021235] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.024126] PCI: CLS 0 bytes, default 64 [ 1.026635] Unpacking initramfs... [ 2.434665] debug: unmapping init [mem 0xffff911ebcc54000-0xffff911ebffbffff] [ 2.440061] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.443353] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.447767] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.964664] Initialise system trusted keyrings [ 2.966753] Key type blacklist registered [ 2.968960] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.981146] zbud: loaded [ 2.984734] *** VALIDATE nfs *** [ 2.986209] *** VALIDATE nfs4 *** [ 2.988253] pstore: using deflate compression [ 2.992344] Platform Keyring initialized [ 3.099742] NET: Registered protocol family 38 [ 3.101618] Key type asymmetric registered [ 3.103457] Asymmetric key parser 'x509' registered [ 3.105740] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.109296] io scheduler mq-deadline registered [ 3.111393] io scheduler kyber registered [ 3.113469] io scheduler bfq registered [ 3.115519] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.118891] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.121818] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.124833] ACPI: Power Button [PWRF] [ 3.131393] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.139075] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.152321] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.159992] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.177730] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.206231] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.235217] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.240761] Non-volatile memory driver v1.3 [ 3.242667] Linux agpgart interface v0.103 [ 3.278064] virtio_blk virtio1: [vda] 146728 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.280870] vda: detected capacity change from 0 to 75124736 [ 3.297060] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.300471] vdb: detected capacity change from 0 to 1073741824 [ 3.317644] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.320569] vdc: detected capacity change from 0 to 2621440000 [ 3.334803] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.337727] vdd: detected capacity change from 0 to 2621440000 [ 3.351888] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.354674] vde: detected capacity change from 0 to 4294967296 [ 3.372100] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.375507] vdf: detected capacity change from 0 to 4294967296 [ 3.381553] libphy: Fixed MDIO Bus: probed [ 3.387593] usbcore: registered new interface driver usbserial_generic [ 3.390716] usbserial: USB Serial support registered for generic [ 3.393437] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.397635] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.399875] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.402180] mousedev: PS/2 mouse device common for all mice [ 3.404593] rtc_cmos 00:05: RTC can wake from S4 [ 3.407528] rtc_cmos 00:05: registered as rtc0 [ 3.409801] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.413168] intel_pstate: CPU model not supported [ 3.415732] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.419620] hid: raw HID events driver (C) Jiri Kosina [ 3.422775] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.423134] usbcore: registered new interface driver usbhid [ 3.428073] usbhid: USB HID core driver [ 3.429780] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.430195] drop_monitor: Initializing network drop monitor service [ 3.435624] Initializing XFRM netlink socket [ 3.437833] NET: Registered protocol family 10 [ 3.440848] Segment Routing with IPv6 [ 3.442254] NET: Registered protocol family 17 [ 3.444122] mpls_gso: MPLS GSO support [ 3.450341] RAS: Correctable Errors collector initialized. [ 3.452869] AVX version of gcm_enc/dec engaged. [ 3.454855] AES CTR mode by8 optimization enabled [ 3.531446] sched_clock: Marking stable (3531363446, 0)->(4502555520, -971192074) [ 3.535814] registered taskstats version 1 [ 3.537613] Loading compiled-in X.509 certificates [ 3.539718] zswap: loaded using pool lzo/zbud [ 3.565294] Key type big_key registered [ 3.604406] Key type encrypted registered [ 3.606246] ima: No TPM chip found, activating TPM-bypass! [ 3.608684] ima: Allocated hash algorithm: sha1 [ 3.610120] ima: No architecture policies found [ 3.611600] evm: Initialising EVM extended attributes: [ 3.613417] evm: security.selinux [ 3.614855] evm: security.ima [ 3.616032] evm: security.capability [ 3.617405] evm: HMAC attrs: 0x1 [ 3.623279] rtc_cmos 00:05: setting system clock to 2026-09-08 04:14:39 UTC (1788840879) [ 3.630396] debug: unmapping init [mem 0xffffffff8fe03000-0xffffffff8fffffff] [ 3.633683] debug: unmapping init [mem 0xffffffff8eb82000-0xffffffff8ee58fff] [ 3.643102] Write protecting the kernel read-only data: 28672k [ 3.647294] debug: unmapping init [mem 0xffffffff8d203000-0xffffffff8d3fffff] [ 3.650704] debug: unmapping init [mem 0xffffffff8db14000-0xffffffff8dbfffff] [ 3.692089] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.701120] systemd[1]: Detected virtualization kvm. [ 3.703060] systemd[1]: Detected architecture x86-64. [ 3.704863] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.736919] systemd[1]: No hostname configured. [ 3.739047] systemd[1]: Set hostname to . [ 3.741274] random: systemd: uninitialized urandom read (16 bytes read) [ 3.744138] systemd[1]: Initializing machine ID from random generator. [ 3.786790] random: ln: uninitialized urandom read (6 bytes read) [ 3.868179] random: systemd: uninitialized urandom read (16 bytes read) [ 3.871437] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.876550] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.882099] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Journal Service... Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.477615] device-mapper: uevent: version 1.0.3 [ 4.480187] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.246344] virtio_net virtio0 ens2: renamed from eth0 [ 5.255499] random: fast init done [ 5.315332] scsi host0: ata_piix [ 5.331729] scsi host1: ata_piix [ 5.337830] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.340959] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.687472] dracut-initqueue[588]: RTNETLINK answers: File exists [ 10.125534] random: crng init done [ 10.127541] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.554372] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.713207] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.003835] SELinux: Disabled at runtime. [ 12.067661] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.075991] systemd[1]: Detected virtualization kvm. [ 12.077796] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.593851] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.597195] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.601139] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.608360] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.611818] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.619410] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.623339] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Slices. Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Starting udev Co[ 12.720481] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS ldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target RPC Port Mapper. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.146129] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.435226] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.443441] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.626215] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.635858] EDAC sbridge: Ver: 1.1.2 [ 14.958187] Key type dns_resolver registered [ 15.270418] NFS: Registering the id_resolver key type [ 15.273767] Key type id_resolver registered [ 15.275565] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg311-server login: [ 39.283436] hrtimer: interrupt took 2707053 ns [ 46.915752] libcfs: loading out-of-tree module taints kernel. [ 46.952848] Key type ._llcrypt registered [ 46.960323] Key type .llcrypt registered [ 47.129701] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 65.924147] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 67.364857] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 67.382829] alg: No test for adler32 (adler32-zlib) [ 68.843400] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 69.853327] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 71.688174] Key type lgssc registered [ 73.758565] Lustre: Echo OBD driver; http://www.lustre.org/ [ 88.298787] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 133.815229] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 148.383926] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 148.427231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 149.845952] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 149.886311] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 150.048767] Lustre: lustre-MDT0000: new disk, initializing [ 150.151115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 150.180553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 154.624444] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 168.519921] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 168.659963] Lustre: 6522:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 168.704663] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 168.709350] Lustre: Skipped 1 previous similar message [ 168.860101] Lustre: lustre-MDT0001: new disk, initializing [ 168.938398] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 168.966497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 168.973933] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 173.278951] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 178.169739] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 189.122992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 189.483979] Lustre: lustre-OST0000: new disk, initializing [ 189.489762] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 189.505808] Lustre: 8457:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 189.587244] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.542595] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 197.138754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 197.149813] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 197.241064] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 210.288297] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 210.401800] Lustre: lustre-OST0001: new disk, initializing [ 210.409492] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 210.415361] Lustre: 9528:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 210.486980] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 217.605913] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 220.214990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 220.228951] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 220.294649] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 229.858304] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 239.697067] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 246.926790] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing check_logdir /tmp/testlogs/ [ 252.985778] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing yml_node [ 257.318659] Lustre: DEBUG MARKER: Client: 2.17.57.114 [ 259.869810] Lustre: DEBUG MARKER: MDS: 2.17.57.114 [ 262.264524] Lustre: DEBUG MARKER: OSS: 2.17.57.114 [ 263.769419] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 8 00:18:57 EDT 2026 [ 281.310416] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 283.134591] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 302.048776] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 302.055966] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 302.058929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 302.067657] Lustre: Skipped 3 previous similar messages [ 302.086632] Lustre: Skipped 3 previous similar messages [ 307.169786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 307.173962] Lustre: Skipped 3 previous similar messages [ 312.289042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.336296] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 314.515763] Lustre: server umount lustre-MDT0000 complete [ 318.117863] LustreError: 8456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841193 with bad export cookie 11025484515671301763 [ 318.123591] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 318.133698] LustreError: 8456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 322.528574] LustreError: 6527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 322.531482] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 322.542168] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 322.542177] Lustre: Skipped 3 previous similar messages [ 322.546439] LustreError: 6527:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 322.564430] Lustre: Skipped 1 previous similar message [ 323.554312] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 323.569134] LustreError: 6526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 324.809433] Lustre: server umount lustre-MDT0001 complete [ 329.684161] Lustre: server umount lustre-OST0000 complete [ 334.517732] Lustre: server umount lustre-OST0001 complete [ 342.482438] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 352.134626] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 391.030424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 391.324529] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 391.350402] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 391.450862] Lustre: lustre-MDT0000: new disk, initializing [ 391.544127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 391.571710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 395.663276] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 406.337401] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 406.597567] Lustre: 15085:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 406.651608] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 406.657990] Lustre: Skipped 1 previous similar message [ 406.750972] Lustre: lustre-MDT0001: new disk, initializing [ 406.923554] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 406.969539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 406.980059] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 411.587608] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 420.900846] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 426.147978] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 434.145512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 434.602826] Lustre: lustre-OST0000: new disk, initializing [ 434.606303] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 434.614790] Lustre: 17052:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 434.766166] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 436.636878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 436.657527] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 436.897994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 442.560852] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 454.404266] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 459.561811] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 459.784493] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 463.718645] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 463.961253] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 467.426271] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 467.438510] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 467.446292] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 467.446300] Lustre: Skipped 3 previous similar messages [ 467.473633] Lustre: Skipped 1 previous similar message [ 472.344269] Lustre: server umount lustre-OST0000 complete [ 477.666104] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 477.673340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 477.688299] Lustre: Skipped 1 previous similar message [ 477.701235] Lustre: Skipped 2 previous similar messages [ 482.977214] Lustre: server umount lustre-MDT0000 complete [ 487.392614] LustreError: 15880:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841363 with bad export cookie 11025484515671303296 [ 487.403316] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 487.413121] LustreError: 15880:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 487.948672] Lustre: server umount lustre-MDT0001 complete [ 493.318960] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 00:22:47 (1788841367) [ 501.191879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 501.860664] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.923539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 506.439824] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 507.369913] LustreError: 19318:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 511.461718] LustreError: 19317:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.283372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 523.632871] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 531.859525] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 538.046542] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 545.273263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 552.734284] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 561.067811] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 568.072751] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 578.532465] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 578.535300] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 578.559529] Lustre: Skipped 2 previous similar messages [ 578.576517] Lustre: Skipped 3 previous similar messages [ 581.592260] Lustre: server umount lustre-MDT0000 complete [ 583.653076] LustreError: 19314:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 583.678106] LustreError: 19314:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 585.578425] LustreError: 19298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841461 with bad export cookie 11025484515671304178 [ 585.588136] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 585.589918] LustreError: 19298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 586.011276] Lustre: server umount lustre-MDT0001 complete [ 599.560974] Lustre: server umount lustre-OST0000 complete [ 622.219244] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 633.113303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 633.610297] LustreError: 23695:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 633.708129] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 633.716289] Lustre: Skipped 2 previous similar messages [ 638.593827] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 648.553329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 653.521318] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 656.511589] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 663.565255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 670.642250] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 679.799396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 679.989350] Lustre: lustre-OST0001: new disk, initializing [ 679.995953] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 680.008396] Lustre: 26014:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 682.013236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 682.030311] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 682.159644] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 687.145648] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 697.240519] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 701.306044] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 715.746505] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 715.757548] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 715.767879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 719.794929] Lustre: server umount lustre-MDT0000 complete [ 723.429279] LustreError: 23695:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 723.480622] LustreError: 23695:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 724.082580] LustreError: 24834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841599 with bad export cookie 11025484515671305025 [ 724.091964] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 724.097190] LustreError: 24834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 724.638752] Lustre: server umount lustre-MDT0001 complete [ 738.318985] Lustre: server umount lustre-OST0000 complete [ 752.775412] Lustre: server umount lustre-OST0001 complete [ 774.140336] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 785.509796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 786.207631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 786.221644] Lustre: Skipped 3 previous similar messages [ 791.529378] LustreError: 29037:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 791.543919] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 791.555399] LustreError: 29037:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 800.985113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 806.468645] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 809.616652] Lustre: 30185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 817.007497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 823.783621] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 832.192575] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 838.437474] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 845.640451] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 849.476892] Lustre: 32061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 858.086697] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 858.099416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 858.105153] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 858.116810] Lustre: Skipped 6 previous similar messages [ 858.160059] Lustre: Skipped 7 previous similar messages [ 862.174935] Lustre: server umount lustre-MDT0000 complete [ 866.254270] LustreError: 29017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841742 with bad export cookie 11025484515671306320 [ 866.270708] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 866.501400] Lustre: server umount lustre-MDT0001 complete [ 880.951525] Lustre: server umount lustre-OST0000 complete [ 894.911563] Lustre: server umount lustre-OST0001 complete [ 903.606763] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 00:29:37 (1788841777) [ 937.084260] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 947.917948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 948.351476] LustreError: 34866:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 948.374098] LustreError: 34866:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 953.935330] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 963.480492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 968.872348] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 972.262058] Lustre: 36013:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 978.929792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 985.734209] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 995.283499] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1002.796369] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1010.503675] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1014.325887] Lustre: 37893:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1020.387559] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1020.404463] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1020.429174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1026.239959] Lustre: server umount lustre-MDT0000 complete [ 1030.268411] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788841906 with bad export cookie 11025484515671307426 [ 1030.273086] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1030.288251] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1030.801938] Lustre: server umount lustre-MDT0001 complete [ 1045.757515] Lustre: server umount lustre-OST0000 complete [ 1048.032557] Lustre: 3653:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788841907/real 1788841907] req@ffff911e03edce00 x1875735683092224/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788841923 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1049.786206] Lustre: server umount lustre-OST0001 complete [ 1059.049488] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 00:32:12 (1788841932) [ 1075.477657] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 1085.742659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1086.251607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1086.265913] Lustre: Skipped 7 previous similar messages [ 1091.070584] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1100.005498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1105.055936] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1108.092616] Lustre: 41182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1115.339308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1122.216646] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1131.788146] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1139.130916] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1148.030214] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1153.561426] Lustre: 43060:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1155.066429] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1155.069088] Lustre: Skipped 1 previous similar message [ 1161.701659] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1161.709066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1161.726817] Lustre: Skipped 4 previous similar messages [ 1164.756150] Lustre: server umount lustre-MDT0000 complete [ 1169.416525] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842045 with bad export cookie 11025484515671308532 [ 1169.442066] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1175.930226] Lustre: server umount lustre-MDT0001 complete [ 1184.665284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1201.470562] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1205.226995] LustreError: 43933:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1205.231133] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1205.246157] LustreError: 43933:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 29 previous similar messages [ 1210.558633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1215.636773] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1216.493357] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1216.503176] Lustre: Skipped 1 previous similar message [ 1222.665614] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1224.390070] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1229.515514] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1231.066624] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1236.452610] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1242.805102] Lustre: server umount lustre-MDT0000 complete [ 1246.952162] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842122 with bad export cookie 11025484515671309638 [ 1246.955990] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1246.963754] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1247.402338] Lustre: server umount lustre-MDT0001 complete [ 1253.125721] Lustre: server umount lustre-OST0000 complete [ 1257.478235] Lustre: server umount lustre-OST0001 complete [ 1265.766722] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 00:35:39 (1788842139) [ 1282.110923] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 1292.362495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1297.439689] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1306.881657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1312.265522] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1315.537145] Lustre: 48815:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1322.504516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1329.276403] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1337.814682] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1345.589280] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1355.227712] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1361.084782] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1374.178431] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1374.180966] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1374.204203] Lustre: Skipped 11 previous similar messages [ 1374.217230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1374.231088] Lustre: Skipped 23 previous similar messages [ 1376.440889] Lustre: server umount lustre-MDT0000 complete [ 1380.689507] LustreError: 47645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842256 with bad export cookie 11025484515671310457 [ 1380.696741] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1380.711490] LustreError: 47645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1381.624968] Lustre: server umount lustre-MDT0001 complete [ 1396.920376] Lustre: server umount lustre-OST0000 complete [ 1401.824267] Lustre: 3652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788842261/real 1788842261] req@ffff911e05117b80 x1875735683238656/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788842277 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1402.253811] Lustre: server umount lustre-OST0001 complete [ 1413.702615] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 00:38:07 (1788842287) [ 1416.655731] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1419.180544] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 00:38:12 (1788842292) [ 1431.032976] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 1441.283202] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 1467.014394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1467.395581] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1467.581201] Lustre: lustre-MDT0000: new disk, initializing [ 1467.717493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1473.169627] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1482.808819] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1490.042719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1490.312496] Lustre: lustre-OST0000: new disk, initializing [ 1490.325085] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1490.333656] Lustre: Skipped 1 previous similar message [ 1490.341144] Lustre: 54893:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1491.636567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1491.652387] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1491.707198] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1497.296449] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1507.890913] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1511.072823] Lustre: DEBUG MARKER: create test files [ 1548.257728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1553.303613] Lustre: server umount lustre-OST0000 complete [ 1557.529749] Lustre: server umount lustre-MDT0000 complete [ 1569.799692] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 1572.723618] Key type lgssc unregistered [ 1572.978234] LNet: 56521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1573.002834] LNetError: 56521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1573.029267] LNet: Removed LNI 192.168.203.111@tcp [ 1574.144416] Key type .llcrypt unregistered [ 1574.150538] Key type ._llcrypt unregistered [ 1601.111025] Key type ._llcrypt registered [ 1601.113015] Key type .llcrypt registered [ 1601.225651] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 1601.924571] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1601.936904] alg: No test for adler32 (adler32-zlib) [ 1603.024206] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 1603.256463] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 1604.936398] Key type lgssc registered [ 1606.168704] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1613.644372] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1613.673119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1615.264804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1620.136846] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1628.821611] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1635.365309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1635.761805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1640.890572] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1640.995420] Lustre: 59232:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1641.006410] Lustre: 59232:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1641.013911] Lustre: 59232:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1641.023341] Lustre: 59232:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1641.031179] Lustre: 59232:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1641.038309] Lustre: 59232:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1642.522802] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1650.337478] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1655.728063] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1661.557881] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1697.250360] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1697.255097] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1697.295560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1699.841081] Lustre: server umount lustre-OST0000 complete [ 1704.574284] Lustre: server umount lustre-MDT0000 complete [ 1717.314721] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 1719.945802] Key type lgssc unregistered [ 1720.244952] LNet: 60867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1720.253480] LNetError: 60867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1721.321272] LNet: Removed LNI 192.168.203.111@tcp [ 1722.061431] Key type .llcrypt unregistered [ 1722.064322] Key type ._llcrypt unregistered [ 1751.519733] Key type ._llcrypt registered [ 1751.524366] Key type .llcrypt registered [ 1751.698509] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 1752.846985] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1752.892358] alg: No test for adler32 (adler32-zlib) [ 1754.021593] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 1754.334821] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 1756.056159] Key type lgssc registered [ 1757.312767] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1766.060254] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1766.076383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1767.521035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1773.417617] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1782.921438] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1789.762565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1790.136781] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1792.187494] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1792.245599] Lustre: 63697:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1792.257911] Lustre: 63697:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1792.262686] Lustre: 63697:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1792.267310] Lustre: 63697:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1792.272064] Lustre: 63697:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1792.277173] Lustre: 63697:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1795.608904] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1803.388783] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1809.780849] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1823.202331] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1823.233201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1825.626589] Lustre: server umount lustre-OST0000 complete [ 1830.120684] Lustre: server umount lustre-MDT0000 complete [ 1842.998369] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 1845.654068] Key type lgssc unregistered [ 1845.983349] LNet: 65256:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1845.989912] LNetError: 65256:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1846.021206] LNet: Removed LNI 192.168.203.111@tcp [ 1846.975540] Key type .llcrypt unregistered [ 1846.979477] Key type ._llcrypt unregistered [ 1867.023230] Key type ._llcrypt registered [ 1867.025757] Key type .llcrypt registered [ 1867.198216] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 1885.307945] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 1886.431329] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1886.448632] alg: No test for adler32 (adler32-zlib) [ 1887.560993] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 1887.847821] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 1889.576245] Key type lgssc registered [ 1891.015799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1929.224585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1929.281764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1930.584267] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1930.636903] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1930.779551] Lustre: lustre-MDT0000: new disk, initializing [ 1930.941081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1930.961239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1936.513170] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1948.532588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1948.681378] Lustre: 68412:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1948.746091] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1948.750572] Lustre: Skipped 1 previous similar message [ 1948.836948] Lustre: lustre-MDT0001: new disk, initializing [ 1949.027111] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1949.135582] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1949.155650] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1955.038183] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1965.630203] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1970.851608] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1977.556835] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1977.816629] Lustre: lustre-OST0000: new disk, initializing [ 1977.822272] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1977.827609] Lustre: 70382:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1977.914336] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1983.548069] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1983.575901] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1983.696879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1984.069138] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 1994.014478] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1999.031733] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1999.313709] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2004.827702] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2005.203488] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2009.588134] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2009.594214] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2009.600409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2009.609644] Lustre: Skipped 1 previous similar message [ 2014.693874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2014.714896] Lustre: Skipped 2 previous similar messages [ 2019.824276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2019.828483] Lustre: Skipped 1 previous similar message [ 2021.344136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2021.561601] Lustre: server umount lustre-OST0000 complete [ 2025.954263] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2025.967434] LustreError: Skipped 1 previous similar message [ 2025.972728] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2025.998987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2030.050757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2030.055188] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2031.470815] Lustre: server umount lustre-MDT0000 complete [ 2035.673282] LustreError: 68404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788842911 with bad export cookie 1666291251010371094 [ 2035.678620] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2035.690279] LustreError: 68404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2036.117902] Lustre: server umount lustre-MDT0001 complete [ 2046.762500] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 00:48:40 (1788842920) [ 2059.433599] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 2100.149646] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 2109.245042] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 2153.798786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2154.114388] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2154.156670] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2154.268116] Lustre: lustre-MDT0000: new disk, initializing [ 2154.380813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2154.401625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2158.821489] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2170.477841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2170.713454] Lustre: 76409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2170.822454] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2170.837498] Lustre: Skipped 1 previous similar message [ 2170.988558] Lustre: lustre-MDT0001: new disk, initializing [ 2171.197661] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2171.257461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2171.276649] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2176.176374] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2186.350227] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2191.741797] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2198.314460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2198.592374] Lustre: lustre-OST0000: new disk, initializing [ 2198.596632] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2198.606882] Lustre: 78375:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2198.702654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2204.892990] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2205.782776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2205.802542] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2205.930152] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2214.745463] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2219.266368] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2219.458259] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2224.069447] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2224.300369] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2226.658246] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2226.666813] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2226.673785] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2226.685714] Lustre: Skipped 1 previous similar message [ 2226.690016] Lustre: Skipped 1 previous similar message [ 2232.447317] Lustre: server umount lustre-OST0000 complete [ 2236.910683] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2236.924809] Lustre: Skipped 1 previous similar message [ 2242.277046] Lustre: server umount lustre-MDT0000 complete [ 2245.928879] LustreError: 80118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843121 with bad export cookie 1666291251010371976 [ 2245.930346] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2245.950248] LustreError: 80118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2246.243759] Lustre: server umount lustre-MDT0001 complete [ 2255.264598] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 00:52:08 (1788843128) [ 2263.034297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2263.514524] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2263.531262] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2263.615402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2268.643143] LustreError: 80789:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2269.057763] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2273.762412] LustreError: 80788:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2278.899694] LustreError: 80789:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2279.348605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2284.082394] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2292.440381] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2298.159441] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2304.761674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2305.171221] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2305.184838] Lustre: Skipped 1 previous similar message [ 2312.389687] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2320.242971] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2323.641386] Lustre: 83461:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting b15e2c16-bcf7-4d2f-b5fe-831a84f4044f at adminstrative request [ 2324.962420] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2324.970352] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2334.195232] Lustre: MGS: Client b15e2c16-bcf7-4d2f-b5fe-831a84f4044f (at 0@lo) reconnecting [ 2334.209231] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x171fdb107cd084fa to 0x171fdb107cd0879a [ 2334.221450] Lustre: MGC192.168.203.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2362.215349] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2362.487407] Lustre: lustre-OST0001: new disk, initializing [ 2362.526867] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2362.544287] Lustre: 83854:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2369.955470] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2370.327444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2370.365439] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2370.578233] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2380.269485] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2385.901660] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2385.905883] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2385.915600] LustreError: Skipped 2 previous similar messages [ 2385.923286] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2385.938348] Lustre: Skipped 1 previous similar message [ 2385.950284] Lustre: Skipped 8 previous similar messages [ 2389.868688] Lustre: server umount lustre-OST0001 complete [ 2391.037114] LustreError: 82612:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2391.057452] LustreError: 82612:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2400.072473] Lustre: server umount lustre-OST0000 complete [ 2406.381221] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2406.392792] Lustre: Skipped 3 previous similar messages [ 2410.080497] Lustre: server umount lustre-MDT0000 complete [ 2411.491443] LustreError: 80789:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2411.516065] LustreError: 80789:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2414.078413] LustreError: 81527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843289 with bad export cookie 1666291251010373530 [ 2414.098550] LustreError: 81527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2414.105103] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2414.415149] Lustre: server umount lustre-MDT0001 complete [ 2424.069854] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 00:54:57 (1788843297) [ 2426.014219] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2428.503477] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 00:55:01 (1788843301) [ 2430.913685] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2432.865562] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 00:55:06 (1788843306) [ 2434.731680] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2436.390302] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 00:55:10 (1788843310) [ 2437.898363] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2440.087137] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 00:55:13 (1788843313) [ 2442.111131] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2447.666076] loop: module loaded [ 2447.682956] loop0: detected capacity change from 0 to 4294967296 [ 2447.743361] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2449.499796] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2453.232479] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2457.412655] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2484.664616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2484.699772] LustreError: 87981:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2484.707541] LustreError: 87981:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2484.718401] LustreError: 87981:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2484.730484] LustreError: 87981:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2504.491591] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 2514.835883] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 2552.885600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2553.260582] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2553.294770] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2553.377392] Lustre: lustre-MDT0000: new disk, initializing [ 2553.478033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2553.487545] Lustre: Skipped 1 previous similar message [ 2553.503826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2559.168955] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2571.632513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2571.767412] Lustre: 90848:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2572.087041] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2577.034243] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2587.340942] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2593.460251] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2601.378257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2601.634973] Lustre: 92816:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2607.705737] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2607.832232] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2608.413546] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2617.937513] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2622.368786] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2622.598644] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2627.164478] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2627.409827] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2633.700922] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2633.703738] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2633.739073] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2633.752565] LustreError: Skipped 1 previous similar message [ 2633.762546] Lustre: Skipped 5 previous similar messages [ 2636.126207] Lustre: server umount lustre-OST0000 complete [ 2643.943419] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2646.409183] Lustre: server umount lustre-MDT0000 complete [ 2649.057631] LustreError: 90859:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2649.075466] LustreError: 90859:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2650.227244] LustreError: 91642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843526 with bad export cookie 1666291251010374146 [ 2650.232232] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2650.254491] LustreError: 91642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2650.744858] Lustre: server umount lustre-MDT0001 complete [ 2659.580914] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 00:58:53 (1788843533) [ 2666.465862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2671.914196] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2681.065202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2685.839219] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2693.610742] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2700.556360] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2707.802525] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2708.138161] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2708.147484] Lustre: Skipped 4 previous similar messages [ 2714.705984] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2722.922768] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2728.200959] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2728.377515] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2733.373673] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2733.623350] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2741.003098] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2741.323290] Lustre: lustre-OST0001: new disk, initializing [ 2741.328666] Lustre: Skipped 2 previous similar messages [ 2741.334935] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2741.346424] Lustre: Skipped 3 previous similar messages [ 2741.352294] Lustre: 98600:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2746.998746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2747.013253] Lustre: Skipped 2 previous similar messages [ 2747.032432] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2747.225731] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2748.829042] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2759.254886] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2764.703846] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2764.963730] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2770.243170] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2770.531564] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2789.990613] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2817.932281] Lustre: Failing over lustre-MDT0000 [ 2818.490677] Lustre: server umount lustre-MDT0000 complete [ 2819.048520] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2819.052987] LustreError: 100056:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2819.060041] Lustre: Skipped 4 previous similar messages [ 2819.096727] LustreError: 100056:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 2834.402379] Lustre: 66635:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788843694/real 1788843694] req@0000000023b17188 x1875737589333504/t0(0) o400->MGC192.168.203.111@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788843710 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2834.444155] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2844.007166] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2844.010839] LDISKFS-fs (dm-0): recovery complete [ 2844.025309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2844.641248] LustreError: 66633:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000e42d039f x1875737589341440/t0(0) o250->MGC192.168.203.111@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2844.969483] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2845.750515] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2845.756855] Lustre: 100056:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2850.181734] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 2850.277529] Lustre: 96470:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2850.295958] Lustre: 96470:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2850.316571] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2850.441692] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2850.760297] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2851.099352] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2851.115915] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2851.441050] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2851.449259] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2851.464825] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2851.476518] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2852.142812] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2852.156686] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2852.480140] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2852.494635] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2853.772860] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2853.781157] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2854.419638] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2854.447488] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2854.776262] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2854.785348] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2857.969903] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2857.988583] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2858.626856] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2858.638640] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2858.960748] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2858.983634] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2865.978601] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2865.988913] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2866.635259] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2866.645848] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2866.976146] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2866.997653] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2882.272996] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2882.291812] Lustre: 101468:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2882.641188] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2882.655331] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2882.984227] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2882.992174] LustreError: 101468:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2905.256151] Lustre: 101468:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2905.269597] Lustre: 101468:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2905.277136] Lustre: 101468:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client c7670bfb-2f60-45f8-a217-85acb3ff2c14@192.168.203.11@tcp [ 2905.290435] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2905.297970] Lustre: 101468:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@0000000017a5d8e8 x1875737570248832/t0(8589934768) o35->c7670bfb-2f60-45f8-a217-85acb3ff2c14@192.168.203.11@tcp:396/0 lens 392/0 e 0 to 0 dl 1788843791 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2905.332898] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2905.368201] Lustre: lustre-MDT0000: Denying connection for new client c7670bfb-2f60-45f8-a217-85acb3ff2c14 (at 192.168.203.11@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2905.369330] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2905.425191] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2905.432419] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2905.439961] Lustre: Skipped 2 previous similar messages [ 2905.481225] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2905.484744] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2921.445776] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2921.447932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2921.455215] LustreError: Skipped 1 previous similar message [ 2921.476728] Lustre: Skipped 4 previous similar messages [ 2927.644090] Lustre: server umount lustre-OST0000 complete [ 2932.201906] LustreError: 97051:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2932.240969] LustreError: 97051:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 33 previous similar messages [ 2938.872070] Lustre: server umount lustre-OST0001 complete [ 2947.557116] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2947.567621] Lustre: Skipped 7 previous similar messages [ 2949.636655] Lustre: server umount lustre-MDT0000 complete [ 2953.073052] LustreError: 95211:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788843828 with bad export cookie 1666291251010425358 [ 2953.081260] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2953.089833] LustreError: 95211:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2953.433463] Lustre: server umount lustre-MDT0001 complete [ 2962.267617] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:03:55 (1788843835) [ 2970.906383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2970.911716] Lustre: *** cfs_fail_loc=197, val=0*** [ 2986.465566] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2991.584587] LustreError: 103438:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.111@tcp: failed processing log, type 4: rc = -110 [ 2996.733610] Lustre: *** cfs_fail_loc=197, val=0*** [ 2996.741493] Lustre: Skipped 65 previous similar messages [ 3017.313152] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3017.317314] Lustre: Skipped 2 previous similar messages [ 3023.347645] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3031.479642] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3033.263096] Lustre: server umount lustre-OST0000 complete [ 3042.632931] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:05:16 (1788843916) [ 3066.146583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3066.587177] LustreError: 105059:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3066.610035] LustreError: 105059:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3070.877916] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3081.139456] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3085.507307] LustreError: 105090:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3085.521460] LustreError: 105090:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3085.534195] LustreError: 105090:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3085.965621] Lustre: server umount lustre-MDT0000 complete [ 3121.056607] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 3133.070347] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 3176.642903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3176.843296] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3176.861582] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3176.919875] Lustre: lustre-MDT0000: new disk, initializing [ 3176.995706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3182.265540] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3192.522394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3192.629721] Lustre: 109273:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3192.866692] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3196.861755] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3206.548637] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3212.781580] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3219.855966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3220.132253] Lustre: 111241:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3226.694985] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3230.410177] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3236.028350] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3240.847747] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3241.059836] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3245.773435] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3245.942544] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3251.172664] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3251.180706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3251.192385] LustreError: Skipped 4 previous similar messages [ 3251.193168] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3251.204920] Lustre: Skipped 1 previous similar message [ 3251.240662] Lustre: Skipped 10 previous similar messages [ 3254.184115] Lustre: server umount lustre-OST0000 complete [ 3264.823741] Lustre: server umount lustre-MDT0000 complete [ 3269.148101] LustreError: 109265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844145 with bad export cookie 1666291251010435928 [ 3269.149603] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3269.168230] LustreError: 109265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3269.660302] Lustre: server umount lustre-MDT0001 complete [ 3280.925687] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:09:14 (1788844154) [ 3289.336422] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 3292.001069] Key type lgssc unregistered [ 3292.443830] LNet: 113728:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3292.449979] LNetError: 113728:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3292.475381] LNet: Removed LNI 192.168.203.111@tcp [ 3293.764733] Key type .llcrypt unregistered [ 3293.777955] Key type ._llcrypt unregistered [ 3312.660920] Key type ._llcrypt registered [ 3312.664844] Key type .llcrypt registered [ 3312.823182] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 3330.593699] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 3332.227762] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3332.245593] alg: No test for adler32 (adler32-zlib) [ 3333.366699] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 3333.690680] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 3335.448213] Key type lgssc registered [ 3336.661582] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3379.332961] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3379.391431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3380.686260] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3380.745702] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3380.876209] Lustre: lustre-MDT0000: new disk, initializing [ 3380.962128] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3380.991427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3385.360383] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3396.034343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3396.397248] Lustre: lustre-OST0000: new disk, initializing [ 3396.401633] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3396.408698] Lustre: Skipped 1 previous similar message [ 3396.413494] Lustre: 117899:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3396.491318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3402.767384] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3402.870667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3402.890314] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3402.925430] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3413.525141] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3413.701839] Lustre: lustre-OST0001: new disk, initializing [ 3413.705961] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3413.713150] Lustre: 118762:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3413.815527] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3419.944142] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3422.332037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3422.338620] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3422.365848] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3432.422559] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3432.428587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3432.437266] Lustre: Skipped 1 previous similar message [ 3432.446799] Lustre: Skipped 1 previous similar message [ 3434.710711] Lustre: server umount lustre-MDT0000 complete [ 3441.179677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3446.832652] LustreError: 118761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844322 with bad export cookie 15595338800360639010 [ 3446.855284] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3446.856272] LustreError: 118761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3446.990573] Lustre: server umount lustre-OST0000 complete [ 3451.877550] Lustre: server umount lustre-OST0001 complete [ 3461.698725] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:12:15 (1788844335) [ 3484.615945] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:12:38 (1788844358) [ 3495.855500] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 3518.185230] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:13:11 (1788844391) [ 3558.124289] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 3570.651637] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 3608.036043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3608.223711] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3608.276730] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3608.362580] Lustre: lustre-MDT0000: new disk, initializing [ 3608.434658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3608.460531] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3613.188430] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3624.071777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3624.237071] Lustre: 126514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3624.321707] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3624.329239] Lustre: Skipped 1 previous similar message [ 3624.391742] Lustre: lustre-MDT0001: new disk, initializing [ 3624.487940] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3624.538976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3624.562223] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3628.900901] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3638.414712] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3643.685552] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3650.370519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3650.583275] Lustre: lustre-OST0000: new disk, initializing [ 3650.589838] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3650.601891] Lustre: 128481:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3650.713878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3657.225732] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3657.278241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3657.289556] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3657.410625] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3667.127647] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3672.171766] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3672.390435] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3677.613453] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3677.900757] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3683.305547] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3683.312952] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3683.315080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3683.338600] Lustre: Skipped 1 previous similar message [ 3685.870327] Lustre: server umount lustre-OST0000 complete [ 3690.982162] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3690.999156] LustreError: Skipped 1 previous similar message [ 3691.007760] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3691.030534] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3691.037539] Lustre: Skipped 1 previous similar message [ 3693.538243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3693.540662] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3698.658828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3698.668865] Lustre: Skipped 2 previous similar messages [ 3704.289647] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3704.677900] Lustre: server umount lustre-MDT0000 complete [ 3708.535795] LustreError: 126507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844584 with bad export cookie 15595338800360640235 [ 3708.542729] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3708.571721] LustreError: 126507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3709.146857] Lustre: server umount lustre-MDT0001 complete [ 3717.988114] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:16:31 (1788844591) [ 3724.809747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3725.538698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3730.468173] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3730.924022] LustreError: 130896:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3730.939076] LustreError: 130896:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3736.033215] LustreError: 130895:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3738.934460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3743.846576] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3751.537264] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3757.273726] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3763.919116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3764.141784] LustreError: 132717:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3764.302541] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3764.311069] Lustre: Skipped 1 previous similar message [ 3770.202988] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3777.170367] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3782.102302] Lustre: *** cfs_fail_loc=159, val=0*** [ 3798.645614] Lustre: lustre-MDT0000: Client 224a5f2a-7051-4522-9ad3-9ebf242f267f (at 192.168.203.11@tcp) reconnecting [ 3800.164891] Lustre: *** cfs_fail_loc=159, val=0*** [ 3800.185184] Lustre: Skipped 5 previous similar messages [ 3816.532828] Lustre: lustre-MDT0000: Client 224a5f2a-7051-4522-9ad3-9ebf242f267f (at 192.168.203.11@tcp) reconnecting [ 3821.538430] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3821.551383] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3821.555022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3821.555029] Lustre: Skipped 2 previous similar messages [ 3821.585866] Lustre: Skipped 1 previous similar message [ 3823.079037] Lustre: server umount lustre-OST0000 complete [ 3831.781278] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3831.784738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3833.096608] Lustre: server umount lustre-MDT0000 complete [ 3836.905110] LustreError: 130895:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3836.927457] LustreError: 130895:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3837.620551] LustreError: 130877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844713 with bad export cookie 15595338800360641117 [ 3837.621821] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3837.626455] LustreError: 130877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3837.968760] Lustre: server umount lustre-MDT0001 complete [ 3848.140756] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 3850.549114] Key type lgssc unregistered [ 3850.794803] LNet: 134666:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3850.799709] LNetError: 134666:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3851.815935] LNet: Removed LNI 192.168.203.111@tcp [ 3852.563080] Key type .llcrypt unregistered [ 3852.565634] Key type ._llcrypt unregistered [ 3874.755076] Key type ._llcrypt registered [ 3874.757350] Key type .llcrypt registered [ 3874.860656] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 3875.969082] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3876.070758] alg: No test for adler32 (adler32-zlib) [ 3877.332289] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 3877.565553] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 3879.192169] Key type lgssc registered [ 3880.686844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3889.937642] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3889.963496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3891.763627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3896.730146] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3901.926177] LustreError: 136077:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3901.954628] LustreError: 136077:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3903.982042] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3904.462184] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3908.165861] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3915.005208] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3920.341270] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3927.240472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3927.776805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3934.793538] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 3937.094692] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3942.687706] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3951.340608] Lustre: *** cfs_fail_loc=159, val=0*** [ 3967.572502] Lustre: lustre-MDT0000: Client 061b6293-63a4-4b65-9fce-8fc51cf2006d (at 192.168.203.11@tcp) reconnecting [ 3969.793545] Lustre: *** cfs_fail_loc=159, val=0*** [ 3969.799416] Lustre: Skipped 1 previous similar message [ 3984.976298] Lustre: lustre-MDT0001: Client 061b6293-63a4-4b65-9fce-8fc51cf2006d (at 192.168.203.11@tcp) reconnecting [ 3986.712025] Lustre: *** cfs_fail_loc=159, val=0*** [ 3986.716657] Lustre: Skipped 2 previous similar messages [ 4002.428450] Lustre: lustre-MDT0001: Client 061b6293-63a4-4b65-9fce-8fc51cf2006d (at 192.168.203.11@tcp) reconnecting [ 4008.318471] Lustre: 139112:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4011.602245] Lustre: *** cfs_fail_loc=159, val=0*** [ 4011.625151] Lustre: Skipped 4 previous similar messages [ 4026.959380] Lustre: lustre-MDT0000: Client 68c52ddc-8023-402f-8a70-473e50ebb90c (at 192.168.203.11@tcp) reconnecting [ 4028.968993] Lustre: *** cfs_fail_loc=159, val=0*** [ 4028.979708] Lustre: Skipped 15 previous similar messages [ 4045.470657] Lustre: lustre-MDT0000: Client 68c52ddc-8023-402f-8a70-473e50ebb90c (at 192.168.203.11@tcp) reconnecting [ 4047.533909] Lustre: 139405:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4052.966849] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4052.966862] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4052.971377] LustreError: Skipped 1 previous similar message [ 4052.977247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4055.011061] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4055.012939] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4055.015622] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4055.023466] Lustre: Skipped 1 previous similar message [ 4055.030976] LustreError: Skipped 1 previous similar message [ 4059.191981] Lustre: server umount lustre-OST0000 complete [ 4063.213524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4063.218963] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4063.243683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4065.254090] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4065.254538] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4065.291805] Lustre: Skipped 1 previous similar message [ 4065.415202] Lustre: server umount lustre-MDT0000 complete [ 4069.439496] LustreError: 139234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788844945 with bad export cookie 8751659465511173036 [ 4069.449116] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4069.455173] LustreError: 139234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4069.782077] Lustre: server umount lustre-MDT0001 complete [ 4082.199486] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 4086.268388] Key type lgssc unregistered [ 4086.811784] LNet: 140456:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4086.823662] LNetError: 140456:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4086.858681] LNet: Removed LNI 192.168.203.111@tcp [ 4087.898070] Key type .llcrypt unregistered [ 4087.899956] Key type ._llcrypt unregistered [ 4115.183700] Key type ._llcrypt registered [ 4115.187558] Key type .llcrypt registered [ 4115.325403] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 4116.850695] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4116.902640] alg: No test for adler32 (adler32-zlib) [ 4118.224964] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 4118.619469] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 4120.416287] Key type lgssc registered [ 4121.828113] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4131.700942] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4131.755097] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4133.375949] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4138.912303] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4143.595069] LustreError: 141872:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4143.619617] LustreError: 141872:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4148.257034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4148.656989] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4153.273087] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4161.148478] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4167.672982] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4174.323311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4174.634533] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4178.754812] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4181.130207] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4188.274717] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4194.177990] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4196.830369] Lustre: 144455:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4199.907580] Lustre: 144505:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4201.416658] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4202.182239] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4205.862551] Lustre: 144650:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4206.772962] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4207.710780] Lustre: 144749:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4214.759688] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4214.765661] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4214.784356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4214.805110] Lustre: Skipped 1 previous similar message [ 4218.669291] Lustre: server umount lustre-OST0000 complete [ 4224.999930] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4225.001400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4225.022542] Lustre: Skipped 2 previous similar messages [ 4228.796718] Lustre: server umount lustre-MDT0000 complete [ 4230.118372] LustreError: 141872:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4230.141093] LustreError: 141872:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4232.051396] LustreError: 141851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845107 with bad export cookie 8031564311788362179 [ 4232.055602] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4232.063900] LustreError: 141851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4232.484967] Lustre: server umount lustre-MDT0001 complete [ 4243.836800] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 4246.309802] Key type lgssc unregistered [ 4246.607959] LNet: 145809:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4246.615451] LNetError: 145809:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4246.638796] LNet: Removed LNI 192.168.203.111@tcp [ 4247.736521] Key type .llcrypt unregistered [ 4247.743124] Key type ._llcrypt unregistered [ 4271.385249] Key type ._llcrypt registered [ 4271.389096] Key type .llcrypt registered [ 4271.502171] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 4272.746049] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4272.828678] alg: No test for adler32 (adler32-zlib) [ 4273.877424] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 4274.127921] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 4275.808205] Key type lgssc registered [ 4276.803974] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4284.638318] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4284.671191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4286.262148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4290.781549] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4296.674322] LustreError: 147219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4296.688163] LustreError: 147219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4299.540411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4299.848123] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4304.016453] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4313.010678] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4319.351403] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4328.301180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4328.858201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4331.981581] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4335.839919] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4344.318086] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4344.859331] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4351.924612] Lustre: *** cfs_fail_loc=159, val=0*** [ 4351.933197] Lustre: Skipped 1 previous similar message [ 4368.487811] Lustre: lustre-MDT0000: Client 3c98f139-88cd-4898-9c17-53d4f3a55254 (at 192.168.203.11@tcp) reconnecting [ 4373.994253] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4373.997287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4374.002388] LustreError: Skipped 1 previous similar message [ 4374.038231] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4375.009029] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4375.009278] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4375.014832] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4375.019408] Lustre: Skipped 1 previous similar message [ 4375.059724] LustreError: Skipped 5 previous similar messages [ 4380.132550] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4380.139327] Lustre: Skipped 1 previous similar message [ 4385.256307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4388.321083] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4388.484434] Lustre: server umount lustre-OST0000 complete [ 4391.905917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4391.919764] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4391.940344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4391.947628] Lustre: Skipped 1 previous similar message [ 4395.492587] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4400.609954] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4400.616034] Lustre: Skipped 3 previous similar messages [ 4406.240205] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4406.461890] Lustre: server umount lustre-MDT0000 complete [ 4409.707951] LustreError: 147198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845285 with bad export cookie 13373495031867023205 [ 4409.716288] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4409.719958] LustreError: 147198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4409.991742] Lustre: server umount lustre-MDT0001 complete [ 4419.880308] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 4422.179440] Key type lgssc unregistered [ 4422.477708] LNet: 150889:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4422.488825] LNetError: 150889:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4422.505669] LNet: Removed LNI 192.168.203.111@tcp [ 4423.422981] Key type .llcrypt unregistered [ 4423.426120] Key type ._llcrypt unregistered [ 4444.499543] Key type ._llcrypt registered [ 4444.501911] Key type .llcrypt registered [ 4444.604351] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 4445.791445] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4445.859385] alg: No test for adler32 (adler32-zlib) [ 4447.009887] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 4447.345675] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 4449.002261] Key type lgssc registered [ 4450.011769] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4457.099039] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4457.135394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4458.601627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4463.229830] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4468.196610] LustreError: 152280:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4468.209793] LustreError: 152280:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4471.700740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4472.134229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4476.812665] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4484.572978] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4490.362858] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4498.398992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4498.680446] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4503.802533] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4506.806634] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4515.320925] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4517.113862] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4535.373753] Lustre: lustre-OST0000: evicting a65f9fde-c18d-45c2-a586-ad5c4b1f6565 (at 192.168.203.11@tcp) by administrative request [ 4565.473370] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4565.488464] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4565.522616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4568.545299] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4568.553630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4568.558055] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4568.575513] Lustre: Skipped 1 previous similar message [ 4573.668263] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4573.674578] Lustre: Skipped 1 previous similar message [ 4578.272105] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4578.557801] Lustre: server umount lustre-OST0000 complete [ 4583.906036] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4583.912629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4583.928937] Lustre: Skipped 1 previous similar message [ 4583.952228] Lustre: Skipped 1 previous similar message [ 4588.948922] Lustre: server umount lustre-MDT0000 complete [ 4589.044956] LustreError: 153011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4589.092886] LustreError: 153011:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4593.239228] LustreError: 152261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845469 with bad export cookie 3996354931355430713 [ 4593.250021] LustreError: 152261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4593.250444] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4593.638502] Lustre: server umount lustre-MDT0001 complete [ 4605.485284] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 4608.591487] Key type lgssc unregistered [ 4608.909186] LNet: 156351:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4608.921535] LNetError: 156351:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4609.958283] LNet: Removed LNI 192.168.203.111@tcp [ 4610.970300] Key type .llcrypt unregistered [ 4610.974632] Key type ._llcrypt unregistered [ 4687.459701] Key type ._llcrypt registered [ 4687.464352] Key type .llcrypt registered [ 4687.607350] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 4688.881874] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4689.185560] alg: No test for adler32 (adler32-zlib) [ 4690.471904] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 4690.889492] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 4692.584248] Key type lgssc registered [ 4693.967883] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4702.446772] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4702.468079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4704.085150] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4709.790546] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4713.442606] LustreError: 158837:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4713.460994] LustreError: 158837:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4718.564185] LustreError: 158838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4718.810527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4719.267336] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4724.862666] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4733.303831] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4739.916180] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4749.162102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4749.627367] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4757.332755] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4764.986264] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4765.720615] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4773.187583] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4785.638935] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4785.657974] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4785.678392] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4788.707696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4788.708240] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4788.728814] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4791.122666] Lustre: server umount lustre-OST0000 complete [ 4795.873577] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4795.882663] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4795.903556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4795.907760] Lustre: Skipped 1 previous similar message [ 4798.945627] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4798.946865] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4798.957745] Lustre: Skipped 1 previous similar message [ 4804.067356] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4804.083407] Lustre: Skipped 1 previous similar message [ 4809.185619] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4809.465834] Lustre: server umount lustre-MDT0000 complete [ 4813.281906] LustreError: 158832:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4813.311096] LustreError: 158832:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4813.364104] LustreError: 158819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845689 with bad export cookie 3600840971416842193 [ 4813.368213] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4813.386936] LustreError: 158819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4813.683565] Lustre: server umount lustre-MDT0001 complete [ 4826.247453] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 4829.257089] Key type lgssc unregistered [ 4829.553804] LNet: 162402:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4829.566352] LNetError: 162402:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4829.592929] LNet: Removed LNI 192.168.203.111@tcp [ 4830.627805] Key type .llcrypt unregistered [ 4830.634392] Key type ._llcrypt unregistered [ 4856.099546] Key type ._llcrypt registered [ 4856.103070] Key type .llcrypt registered [ 4856.218119] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 4857.539469] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4857.571188] alg: No test for adler32 (adler32-zlib) [ 4858.700927] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 4858.926907] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 4860.597000] Key type lgssc registered [ 4861.883712] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4877.276862] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 01:35:50 (1788845750) [ 4893.866275] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 4904.241551] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 4941.189078] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4941.211719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4942.482589] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4942.518776] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4942.615288] Lustre: lustre-MDT0000: new disk, initializing [ 4942.726065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4942.749138] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4946.967521] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4956.642705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4956.804807] Lustre: 166661:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4956.865763] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4956.870602] Lustre: Skipped 1 previous similar message [ 4956.953922] Lustre: lustre-MDT0001: new disk, initializing [ 4957.065213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4957.121252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4957.141724] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4962.009361] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4971.145257] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4976.628822] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4982.599468] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4982.842761] Lustre: lustre-OST0000: new disk, initializing [ 4982.847317] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4982.856060] Lustre: 168630:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4982.928629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4988.467845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4988.487644] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4988.580196] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4989.101641] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 4998.795996] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5009.377848] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5009.382570] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5009.382746] LustreError: Skipped 1 previous similar message [ 5009.401566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5009.408794] Lustre: Skipped 1 previous similar message [ 5014.500553] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5014.506543] Lustre: Skipped 1 previous similar message [ 5015.253580] Lustre: server umount lustre-OST0000 complete [ 5019.617878] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5019.625455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5019.647571] Lustre: Skipped 2 previous similar messages [ 5019.679083] Lustre: Skipped 1 previous similar message [ 5024.738626] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5024.750692] Lustre: Skipped 1 previous similar message [ 5025.580542] Lustre: server umount lustre-MDT0000 complete [ 5030.083791] LustreError: 167456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788845905 with bad export cookie 9452107584715420678 [ 5030.096380] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5030.101093] LustreError: 167456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5030.444749] Lustre: server umount lustre-MDT0001 complete [ 5044.358637] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 5047.371985] Key type lgssc unregistered [ 5047.727857] LNet: 170535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5047.736744] LNetError: 170535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5047.777329] LNet: Removed LNI 192.168.203.111@tcp [ 5048.737423] Key type .llcrypt unregistered [ 5048.741775] Key type ._llcrypt unregistered [ 5072.123947] Key type ._llcrypt registered [ 5072.127177] Key type .llcrypt registered [ 5072.229886] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 5073.171364] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5073.247212] alg: No test for adler32 (adler32-zlib) [ 5074.355898] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 5074.606691] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 5076.312853] Key type lgssc registered [ 5077.286815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5084.367918] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5084.395608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5085.875590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5090.553541] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 5095.396622] LustreError: 171949:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5095.418486] LustreError: 171949:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5098.765793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5099.091299] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5102.754051] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 5109.570702] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5115.188504] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5121.926765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5122.252281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5129.045596] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 5130.427771] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5137.126062] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5145.043612] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5145.072921] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5145.087903] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5145.095359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.104320] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5146.059727] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5146.082172] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5146.103405] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5146.112994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.130444] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5147.175714] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5147.192979] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5147.205939] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5147.210676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.220817] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5147.943172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.913683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.749480] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5149.755257] Lustre: Skipped 2 previous similar messages [ 5149.771982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5149.781792] Lustre: Skipped 2 previous similar messages [ 5149.784531] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5149.786855] Lustre: Skipped 2 previous similar messages [ 5149.791856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.803141] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5149.809750] Lustre: Skipped 2 previous similar messages [ 5150.599950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.385495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.327214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.326812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.388878] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5154.397767] Lustre: Skipped 4 previous similar messages [ 5154.413399] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5154.426352] Lustre: Skipped 4 previous similar messages [ 5154.429711] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5154.439880] Lustre: Skipped 4 previous similar messages [ 5154.443932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.463058] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5154.479621] Lustre: Skipped 4 previous similar messages [ 5155.619681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.807952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.804070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.900141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.914268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.907743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.700517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.542401] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5162.548415] Lustre: Skipped 7 previous similar messages [ 5162.564301] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5162.573314] Lustre: Skipped 7 previous similar messages [ 5162.576829] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5162.585333] Lustre: Skipped 7 previous similar messages [ 5162.590394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.600214] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5162.611118] Lustre: Skipped 7 previous similar messages [ 5163.417858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.243869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.034751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.008958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.099111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.166221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.148828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.057752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.166589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.068981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.350453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.493279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.354137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.225906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.171506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.185903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.032796] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5179.036508] Lustre: Skipped 16 previous similar messages [ 5179.046941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5179.055666] Lustre: Skipped 16 previous similar messages [ 5179.059481] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5179.063373] Lustre: Skipped 16 previous similar messages [ 5179.067338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.078546] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5179.084572] Lustre: Skipped 16 previous similar messages [ 5179.904827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.685731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.646904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.414374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.364396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.173629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.009929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.774421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.617609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.529483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.627203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.558404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.535482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.830948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.112817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.302949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.279319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.592110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.894473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.045910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.260808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.620882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.461969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.461232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.430161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.422342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.500297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.600545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.526085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.567817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.590345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.945174] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5211.950316] Lustre: Skipped 31 previous similar messages [ 5211.981382] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5212.002166] Lustre: Skipped 31 previous similar messages [ 5212.012034] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5212.028393] Lustre: Skipped 31 previous similar messages [ 5212.043493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.070337] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5212.081597] Lustre: Skipped 31 previous similar messages [ 5213.058317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.078593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.356051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.415337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.342967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.380273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.343289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.206955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.028804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.977849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.012514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.918759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.912372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.048917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.028677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.026602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.032997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.245668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.410724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.534251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.311247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.300039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.264327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.193799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.082220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.935879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.812190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.724520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.626730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.496451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.491983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.385183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.347266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.281735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.173611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.170979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.062768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.035468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.026378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.036628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.266818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.180293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.089698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.979563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.018339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.948162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.158531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.944859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.924139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.850546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.806482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.725066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.740864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.805409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.007464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.171165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.406114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.364510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.431572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.475753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.492905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.408247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.497252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.439534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.480440] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5276.491974] Lustre: Skipped 64 previous similar messages [ 5276.508752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.523370] Lustre: Skipped 64 previous similar messages [ 5276.527130] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.533954] Lustre: Skipped 64 previous similar messages [ 5276.545342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.556122] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.561275] Lustre: Skipped 64 previous similar messages [ 5277.334829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.249950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.092791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.996458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.942640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.895372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.929200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.923964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.123393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.213610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.244516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.235918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.138364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.124138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.095750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.996321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.882724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.954594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.820836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.847955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.920741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.091282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.077243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.196524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.111964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.975696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.827796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.759500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.638885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.534645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.379409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.362881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.203769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.978660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.797079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.569796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.393471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.310489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.215402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.154557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.999746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.903824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.834380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.883981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.673167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.641884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.573226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.517835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.255878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.116982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.888684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.706664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.435704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.254649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.097343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.885903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.762594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.653475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.453620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.225722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.949696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.698614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.523740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.562069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.671474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.658950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.716823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.809828] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 5338.818139] LustreError: 171968:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 5338.825279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.890491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.739981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.908600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.059754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.103890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.135130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.178996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.232558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.233485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.324462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.281680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.337303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.235743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.076920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.027646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.978454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.738451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.786479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.741488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.891712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.956782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.028490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.082401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.013870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.918170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.011305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.993139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.919755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.842416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.912950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.019868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.115838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.181288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.166917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.136560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.987172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.962103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.763257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.669038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.540474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.437163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.394558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.546349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.514221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.629671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.948759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.119434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.057960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.220681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.494957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.773931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.017803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.147932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.128489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.114176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.177780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.373233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.507252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.817862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.084543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.407064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.415675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.485872] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5404.491393] Lustre: Skipped 130 previous similar messages [ 5404.532551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5404.558288] Lustre: Skipped 130 previous similar messages [ 5404.565648] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5404.571858] Lustre: Skipped 130 previous similar messages [ 5404.589269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.620993] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5404.631851] Lustre: Skipped 130 previous similar messages [ 5405.755835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.139401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.901293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.924535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.343618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.685859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.291949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.263463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.273643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.264578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.376324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.411790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.433883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.299402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.520188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.588950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.646888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.589789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.754798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.762736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.945310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.150672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.062242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.108117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.976771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.120451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.148898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.184606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.219732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.179209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.189115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.388173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.590783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.594336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.775798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.783050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.878982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.796468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.765061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.712592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.714690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.690543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.819642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.871303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.852130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.899036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.993239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.967807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.922148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.000301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.939204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.917631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.887504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.823442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.646145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.960283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.278839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.366644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.425266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.674518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.581703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.844551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.004899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.898384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.809090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.719917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.847460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.697806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.631840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.510377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.464891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.448126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.579535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.599816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.646910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.758989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.891341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.027379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.175346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.412692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.551588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.678294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.834044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.079370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.058305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.054367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.094264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.100338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.088248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.353610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.623674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.832671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.197511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.350310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.573809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.665511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.838108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.752456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.090189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.180250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.601479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.036376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.620780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.420153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.085560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.453205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.905133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.668161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.802146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.913743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.646820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.800220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.800739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.800563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.731687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.636385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.830023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.761374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.053874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.301955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.570229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.637081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.741761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.567818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.449170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.396205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.607436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.569804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.828801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.628300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.747842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.735367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.775216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.929764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.842123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.915902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.866948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.709870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.669433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.618817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.513419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.685489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.834993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.040545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.045288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.042020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.185935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.265491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.181422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.268293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.263370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.171605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.114750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.002774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.960839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.869059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.963710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.910699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.752665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.606609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.505320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.233730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.102869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.968835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.782747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.583901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.534031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.435136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.303778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.234074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.083488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.937336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.815543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.771982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.652220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.760115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.628030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.527637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.432794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.437433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.217327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.108874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.155185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.012337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.063396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.313898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.468400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.548818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.414716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.381290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.342551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.273851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.258348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.256333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.198634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.124120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.152547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.195135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.082882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.261903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.297608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.493636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.531044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.469201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.621964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.716816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.800454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.665761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.575491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.583854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.471124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.551086] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5633.566228] LustreError: 171968:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5633.576246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.543829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.433380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.337751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.487367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.709956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.804130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.773666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.677376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.669411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.544943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.448604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.495780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.490905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.298960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.200236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.224426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.249192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.401241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.363950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.754404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.806402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.090607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.186666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.327930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.498046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.588611] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5660.594933] Lustre: Skipped 237 previous similar messages [ 5660.608930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5660.620215] Lustre: Skipped 237 previous similar messages [ 5660.623693] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5660.635468] Lustre: Skipped 237 previous similar messages [ 5660.638107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.658325] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5660.670657] Lustre: Skipped 237 previous similar messages [ 5661.827057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.051354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.296862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.332783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.374952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.609941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.521645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.478303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.493388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.668701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.629025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.781409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.824468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.756307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.721732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.570837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.509270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.380896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.479665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.409989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.363497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.293678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.263960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.291879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.149467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.036559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.125094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.200102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.182221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.239757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.378751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.428203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.304785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.180573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.229217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.116470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.019581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.000626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.005773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.219097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.399082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.463862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.518199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.733626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.843523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.682835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.681815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.912906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.023732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.140086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.233160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.271250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.337343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.428127] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5716.443446] LustreError: 171968:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5716.452901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.729917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.060434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.055502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.179191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.286354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.141033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.042899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.078664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.143274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.022572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.976993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.777253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.716717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.542604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.539570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.363266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.389980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.306505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.360749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.392978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.586338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.590982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.637737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.137865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.375763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.334340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.735102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.972575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.139223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.431818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.553601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.592793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.793806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.933949] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5752.941612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.948173] LustreError: 171968:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5753.984086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.994778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.852575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.099251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.187169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.375540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.406460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.349472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.193679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.128269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.062766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.192306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.169564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.454475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.662923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.921732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.111842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.230469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.232991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.400240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.374705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.302763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.038887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.814168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.748430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.800472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.698253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.781944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.883492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.896845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.889837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.103722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.452857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.231985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.454022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.703397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.942965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.825769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.940749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.192783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.151567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.311277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.402562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.329397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.629623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.545753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.960083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.912341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.956532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.018446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.905568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.933369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.888981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.011551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.055987] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5811.069143] LustreError: 171968:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5811.078034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.147936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.263442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.279742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.445395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.401581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.489989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.507038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.527902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.429609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.518807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.628649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.616108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.552488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.515850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.407371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.592245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.653653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.620509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.513863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.634872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.714769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.775829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.758508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.953509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.056647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.143096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.165494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.578461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.880194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.880686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.916627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.952329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.885817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.811332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.710339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.726811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.756081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.668838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.731348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.597502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.549835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.543244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.469221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.587409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.492911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.556573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.449987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.596693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.615074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.712675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.590348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.476658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.345674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.347643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.306180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.315048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.113725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.045373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.882795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.951577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.947435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.905936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.883904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.851542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.842395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.855791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.780403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.790914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.722957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.834869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.850061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.971370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.102475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.973845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.223702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.232639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.372464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.415797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.408994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.471182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.429767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.672691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.598903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.576568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.705768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.783280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.715288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.943445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.972697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.082942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.974909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.907611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.879583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.822251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.704907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.758438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.660056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.743273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.873836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.983128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.170763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.357584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.444408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.569354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.561471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.752892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.823668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.926067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.014510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.093850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.157581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.100624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.149484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.099066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.000270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.988568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.883358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.979436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.807189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.761545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.785470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.710198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.033978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.046729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.074566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.073859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.157912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.220356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.282643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.384395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.316290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.258112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.191445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.014248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.091887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.210075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.137371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.433126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.552213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.525481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.574720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.595938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.574862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.740419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.869126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.219227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.498334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.545681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.821505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.883407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.045411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.245384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.511223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.779710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.030122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.088824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.341511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.618613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.955157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.166362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.483321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.651409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.694083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.889521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.950894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.967855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.900177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.782243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.858078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.022569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.024907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.950238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.806162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.723844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.642443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.573215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.541293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.468634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.289570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.154719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.159169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.993691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.872869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.684487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.633061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.806944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.607968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.722971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.799876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.740721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.702349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.587207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.620982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.449807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.177560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.967349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.959424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.781584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.707662] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6015.717842] LustreError: 171968:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6015.741810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.707163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.605528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.444030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.361278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.187982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.070642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.828929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.627345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.394177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.252704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.049985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.810929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.529533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.322592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.096312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.881274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.674740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.512494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.298642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.161880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.024876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.009076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.923083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.886311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.952331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.062653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.124216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.097847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.241941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.424195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.420069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.653361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.569327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.707902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.912391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.981709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.441596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.458789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.655394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.785194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.012559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.238738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.251029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.431222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.407945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.203287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.191345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.093866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.170074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.259360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.449401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.543427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.818943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.973763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.755704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.841308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.738414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.611526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.518072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.661521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.791584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.895240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.075957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.905146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.796815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.829539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.215945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.491414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.670915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.628560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.799344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.894188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.061787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.148648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.259364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.201541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.262930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.247793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.475759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.403589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.538365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.590362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.496675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.307913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.314810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.362054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.396930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.319910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.277227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.267343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.140292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.996920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.986527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.798381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.860618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.812289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.836415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.632834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.450586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.382767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.414951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.654645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.556423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.377728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.143626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.910608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.876617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.743598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.730714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.689911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.841348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.008639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.454539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.578130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.523300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.642935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.573969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.496479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.319484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.437832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.374130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.427963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.397732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.435618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.395537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.475849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.521342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.432489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.188487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.147364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.995287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.897248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.753586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.714910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.533553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.477259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.573673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.616445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.599833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.512600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.768876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.809428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.641484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.716465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.735448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.915837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.065416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.845660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.775828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.861231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.135213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.265171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.447960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.791480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.204056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.595776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.707177] LustreError: 173771:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6613.722182] LustreError: 171968:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7366.114945] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7366.128598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7366.148648] Lustre: Skipped 498 previous similar messages [ 7366.173384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7366.628891] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7367.149288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7367.162418] Lustre: Skipped 2 previous similar messages [ 7372.165091] Lustre: server umount lustre-OST0000 complete [ 7377.384413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7377.387026] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7377.423680] Lustre: Skipped 1 previous similar message [ 7382.498971] LustreError: 225798:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7383.031379] Lustre: server umount lustre-MDT0000 complete [ 7387.493102] LustreError: 172684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848263 with bad export cookie 14585581712559861096 [ 7387.508355] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7387.516840] LustreError: 172684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7387.896934] Lustre: server umount lustre-MDT0001 complete [ 7400.798467] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 7404.125431] Key type lgssc unregistered [ 7404.442657] LNet: 227252:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7404.457875] LNetError: 227252:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7404.493915] LNet: Removed LNI 192.168.203.111@tcp [ 7405.590580] Key type .llcrypt unregistered [ 7405.594928] Key type ._llcrypt unregistered [ 7435.588211] Key type ._llcrypt registered [ 7435.591228] Key type .llcrypt registered [ 7435.793216] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 7437.396825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7437.615997] alg: No test for adler32 (adler32-zlib) [ 7438.975263] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 7439.246433] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 7441.016885] Key type lgssc registered [ 7442.641064] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7453.412620] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7453.452327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7455.403466] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7462.097738] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7465.961040] LustreError: 228665:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7466.008182] LustreError: 228665:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7471.075478] LustreError: 228666:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7474.266615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7474.647887] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7479.842160] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7487.607237] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7493.966244] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7501.062192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7501.487418] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7507.634887] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7515.970174] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7525.995834] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7530.792244] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7532.589578] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7532.604206] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7532.618978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.633172] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7532.639392] LustreError: 228685:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7535.601150] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7535.614404] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7535.627406] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7536.610423] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7536.621530] Lustre: Skipped 1 previous similar message [ 7540.868541] Lustre: server umount lustre-OST0000 complete [ 7552.688332] LustreError: 228685:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7576.347842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7576.569362] LustreError: 231898:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7576.918565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7582.186762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.224669] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7584.712932] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7586.356874] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.376341] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7592.596452] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7594.698784] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7601.399166] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9612d07a4800.ost_server_uuid 50 [ 7603.773544] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9612d07a4800.ost_server_uuid in IDLE state after 0 sec [ 7611.876870] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7611.881347] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7611.900271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7611.900280] Lustre: Skipped 1 previous similar message [ 7611.968717] Lustre: Skipped 2 previous similar messages [ 7615.844734] Lustre: server umount lustre-OST0000 complete [ 7622.178713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7622.211639] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7622.220091] Lustre: Skipped 2 previous similar messages [ 7627.234276] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7627.249259] Lustre: Skipped 1 previous similar message [ 7627.808966] Lustre: server umount lustre-MDT0000 complete [ 7631.329628] LustreError: 230506:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7631.363844] LustreError: 230506:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7632.945482] LustreError: 228645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848508 with bad export cookie 5689587080289511213 [ 7632.955932] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7632.969968] LustreError: 228645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7633.947926] Lustre: server umount lustre-MDT0001 complete [ 7646.946498] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 7650.659851] Key type lgssc unregistered [ 7651.048351] LNet: 233899:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7651.070546] LNetError: 233899:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7651.097470] LNet: Removed LNI 192.168.203.111@tcp [ 7652.333701] Key type .llcrypt unregistered [ 7652.336711] Key type ._llcrypt unregistered [ 7689.290483] Key type ._llcrypt registered [ 7689.296363] Key type .llcrypt registered [ 7689.459203] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 7691.197430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7691.242872] alg: No test for adler32 (adler32-zlib) [ 7692.485688] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 7692.809844] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 7694.552174] Key type lgssc registered [ 7696.600605] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7709.775707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7709.810951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7711.553594] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7716.466699] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7721.958602] LustreError: 235628:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7721.996411] LustreError: 235628:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7726.663901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7727.089963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7727.283023] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7732.842692] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7736.449533] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7746.367045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7747.201656] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7755.801944] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7756.439705] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7767.078641] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7767.363735] Lustre: lustre-OST0001: new disk, initializing [ 7767.367478] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7767.373214] Lustre: 237938:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7767.452922] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7775.339077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7775.349436] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7775.422750] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7775.552629] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7786.862797] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7795.139558] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7811.564066] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7811.573974] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7811.579302] Lustre: Skipped 1 previous similar message [ 7811.595730] Lustre: Skipped 3 previous similar messages [ 7816.162661] Lustre: server umount lustre-MDT0000 complete [ 7816.680328] LustreError: 236368:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7816.714098] LustreError: 236368:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7819.777997] LustreError: 235610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848695 with bad export cookie 15007817383030123493 [ 7819.783651] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7819.789899] LustreError: 235610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7820.107660] Lustre: server umount lustre-MDT0001 complete [ 7834.159182] Lustre: server umount lustre-OST0000 complete [ 7837.158256] Lustre: 234888:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788848697/real 1788848697] req@ffff911e238e3480 x1875743676135680/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788848713 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7837.196105] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7837.211986] Lustre: Skipped 2 previous similar messages [ 7838.472865] Lustre: server umount lustre-OST0001 complete [ 7865.701511] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 7878.480812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7878.973174] LustreError: 241047:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7879.083547] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7883.523964] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7884.261987] LustreError: 241048:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7884.289175] LustreError: 241048:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7888.360809] LustreError: 241047:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7892.788215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7893.262802] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7898.262441] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7901.627850] Lustre: 242191:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7908.657656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7916.192980] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7918.402168] LustreError: 242543:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7923.533865] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7925.501399] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7925.892136] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7925.904452] Lustre: Skipped 1 previous similar message [ 7933.552605] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 7941.749227] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7945.648073] Lustre: 244060:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7960.035773] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7960.054245] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7960.087460] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7962.094422] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7962.098848] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7962.111428] Lustre: Skipped 2 previous similar messages [ 7962.130155] Lustre: Skipped 3 previous similar messages [ 7963.620809] Lustre: server umount mylustre-MDT0000 complete [ 7967.207041] LustreError: 241044:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7967.229175] LustreError: 241044:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7967.496800] LustreError: 244366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848843 with bad export cookie 15007817383030126300 [ 7967.500104] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7967.527458] LustreError: 244366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7967.858379] Lustre: server umount mylustre-MDT0001 complete [ 7981.629031] Lustre: server umount mylustre-OST0000 complete [ 7995.573457] Lustre: server umount mylustre-OST0001 complete [ 8024.043668] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8036.153193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8036.716199] LustreError: 246371:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8036.816594] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8041.904073] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8051.108646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8055.833791] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8059.076124] Lustre: 247513:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8066.362439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8074.803502] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8082.106924] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 8084.772249] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8090.134942] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8091.973911] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8100.953272] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8105.033200] Lustre: 249383:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8114.385680] Lustre: server umount tfs-MDT0000 complete [ 8115.682877] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8115.706170] LustreError: 246367:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8115.733682] LustreError: 246367:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 8118.302305] LustreError: 248696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788848994 with bad export cookie 15007817383030127966 [ 8118.309178] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8118.318186] LustreError: 248696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8118.815993] Lustre: server umount tfs-MDT0001 complete [ 8133.959215] Lustre: server umount tfs-OST0000 complete [ 8136.930241] Lustre: 234888:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788848996/real 1788848996] req@ffff911f3bbc1180 x1875743676248064/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788849012 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8136.958744] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8136.974462] Lustre: Skipped 3 previous similar messages [ 8137.786170] Lustre: server umount tfs-OST0001 complete [ 8159.562558] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8170.827341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8171.328574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8171.339179] Lustre: Skipped 3 previous similar messages [ 8176.326144] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8186.505538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8191.990171] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8195.219817] Lustre: 252776:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8203.034028] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8210.550206] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8216.786585] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 8220.353191] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8225.793828] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8228.125733] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8237.108698] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8241.035960] Lustre: 254647:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8248.349610] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:32:02 (1788849122) [ 8266.723248] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8266.732398] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8266.755722] Lustre: Skipped 2 previous similar messages [ 8266.789301] Lustre: Skipped 3 previous similar messages [ 8271.854238] LustreError: 251632:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8271.873400] LustreError: 251632:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 8271.967843] Lustre: server umount lustre-MDT0000 complete [ 8275.984309] LustreError: 254650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849151 with bad export cookie 15007817383030129625 [ 8275.989832] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8276.002371] LustreError: 254650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8276.983213] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8276.989432] Lustre: Skipped 1 previous similar message [ 8282.467878] Lustre: server umount lustre-MDT0001 complete [ 8292.668417] Lustre: server umount lustre-OST0000 complete [ 8302.925787] Lustre: server umount lustre-OST0001 complete [ 8310.218329] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 8322.364385] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8361.266886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8361.490988] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8361.522085] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8361.646323] Lustre: lustre-MDT0000: new disk, initializing [ 8361.803764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8361.810955] Lustre: Skipped 3 previous similar messages [ 8361.825459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8366.418813] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8377.244445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8377.456833] Lustre: 257835:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8377.556583] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8377.560830] Lustre: Skipped 1 previous similar message [ 8377.682149] Lustre: lustre-MDT0001: new disk, initializing [ 8377.858112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8377.887555] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8383.538278] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8396.345365] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8396.789840] Lustre: lustre-OST0000: new disk, initializing [ 8396.796120] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8396.802706] Lustre: 259417:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8398.186117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8398.217443] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8398.362361] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8403.826620] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8415.143563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8415.365347] Lustre: lustre-OST0001: new disk, initializing [ 8415.372301] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8415.394712] Lustre: 260293:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8417.058850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8417.077467] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8417.199245] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8422.028279] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8434.787839] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:35:08 (1788849308) [ 8450.101050] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:35:23 (1788849323) [ 8468.457553] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8468.467526] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8468.482339] Lustre: Skipped 4 previous similar messages [ 8468.505304] Lustre: Skipped 3 previous similar messages [ 8471.490399] Lustre: server umount lustre-OST0000 complete [ 8481.772582] Lustre: server umount lustre-MDT0000 complete [ 8485.879227] LustreError: 257827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849361 with bad export cookie 15007817383030131179 [ 8485.895844] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8485.898043] LustreError: 257827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8500.204663] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8500.532616] Lustre: server umount lustre-MDT0001 complete [ 8511.172731] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 8584.186382] Lustre: server umount lustre-OST0001 complete [ 8590.708438] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 8606.916269] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8641.726996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8641.987839] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8642.027369] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8642.192848] Lustre: lustre-MDT0000: new disk, initializing [ 8642.331228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8642.336226] Lustre: Skipped 3 previous similar messages [ 8642.349975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8646.667859] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8657.602638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8657.726981] Lustre: 265145:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8657.949645] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8662.352502] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8671.188638] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8676.803729] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8682.840667] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8683.078502] Lustre: lustre-OST0000: new disk, initializing [ 8683.081345] Lustre: Skipped 1 previous similar message [ 8683.084054] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8683.088349] Lustre: Skipped 2 previous similar messages [ 8683.092252] Lustre: 267113:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8684.909858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8684.921150] Lustre: Skipped 1 previous similar message [ 8684.925168] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8685.050207] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8688.675457] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8697.499613] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8705.518569] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8705.518645] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8705.525376] LustreError: Skipped 1 previous similar message [ 8705.527302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8705.527310] Lustre: Skipped 6 previous similar messages [ 8705.553469] Lustre: Skipped 5 previous similar messages [ 8709.708550] Lustre: server umount lustre-OST0000 complete [ 8714.211958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8719.801334] Lustre: server umount lustre-MDT0000 complete [ 8720.867331] LustreError: 265155:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8720.889592] LustreError: 265155:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 15 previous similar messages [ 8723.446343] LustreError: 265136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849599 with bad export cookie 15007817383030132656 [ 8723.460478] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8723.722225] Lustre: server umount lustre-MDT0001 complete [ 8734.419230] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 8736.801450] Key type lgssc unregistered [ 8737.069460] LNet: 268873:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8737.075253] LNetError: 268873:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8737.086993] LNet: Removed LNI 192.168.203.111@tcp [ 8737.917152] Key type .llcrypt unregistered [ 8737.920095] Key type ._llcrypt unregistered [ 8760.855789] Key type ._llcrypt registered [ 8760.858632] Key type .llcrypt registered [ 8760.959290] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8761.875356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8761.883653] alg: No test for adler32 (adler32-zlib) [ 8762.890372] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 8763.111057] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 8764.768283] Key type lgssc registered [ 8765.724122] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8772.017056] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8772.038822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8773.385027] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8777.269887] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8783.841408] LustreError: 270325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8783.858459] LustreError: 270325:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8784.582024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8784.852541] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8789.095943] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8795.524679] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8800.119769] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8805.912567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8806.241875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8811.472245] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8818.097443] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8818.558448] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8818.977181] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8823.777723] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8823.779970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8823.800049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8823.803704] Lustre: Skipped 1 previous similar message [ 8826.858816] Lustre: server umount lustre-OST0000 complete [ 8830.947385] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8830.957797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8830.968905] Lustre: Skipped 1 previous similar message [ 8830.973297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8834.017208] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8834.018795] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8839.140116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8839.146298] Lustre: Skipped 2 previous similar messages [ 8843.235471] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8843.243163] Lustre: Skipped 1 previous similar message [ 8844.256271] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8844.388762] Lustre: server umount lustre-MDT0000 complete [ 8847.514130] LustreError: 270306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849723 with bad export cookie 15728617145295072762 [ 8847.519543] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8847.526597] LustreError: 270306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8847.850571] Lustre: server umount lustre-MDT0001 complete [ 8856.954883] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 8859.384481] Key type lgssc unregistered [ 8859.650838] LNet: 273909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8859.669098] LNetError: 273909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8859.699789] LNet: Removed LNI 192.168.203.111@tcp [ 8860.506190] Key type .llcrypt unregistered [ 8860.508643] Key type ._llcrypt unregistered [ 8875.892577] Key type ._llcrypt registered [ 8875.896614] Key type .llcrypt registered [ 8875.982586] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8876.801839] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8876.841588] alg: No test for adler32 (adler32-zlib) [ 8877.918558] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 8878.110568] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 8879.770054] Key type lgssc registered [ 8880.639844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8887.823241] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8887.845252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8903.328483] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8908.448544] LustreError: 275100:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.111@tcp: failed processing log, type 4: rc = -110 [ 8935.072376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8939.604232] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 8944.841951] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8946.516601] Lustre: server umount lustre-OST0000 complete [ 8957.995120] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 8959.889571] Key type lgssc unregistered [ 8960.088192] LNet: 276729:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8960.095216] LNetError: 276729:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8960.109803] LNet: Removed LNI 192.168.203.111@tcp [ 8960.659625] Key type .llcrypt unregistered [ 8960.661807] Key type ._llcrypt unregistered [ 8990.897549] Key type ._llcrypt registered [ 8990.900236] Key type .llcrypt registered [ 8990.980249] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 8991.657751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8991.745290] alg: No test for adler32 (adler32-zlib) [ 8992.730081] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 8992.946500] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 8994.592215] Key type lgssc registered [ 8995.499991] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9006.164499] loop0: detected capacity change from 0 to 204800000 [ 9006.339576] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9006.357343] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9006.650876] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9007.431305] loop0: detected capacity change from 0 to 204800000 [ 9007.528190] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9007.536312] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9007.734829] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9008.535319] loop0: detected capacity change from 0 to 204800000 [ 9008.656873] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9008.670918] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9008.744642] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9009.575771] loop0: detected capacity change from 0 to 204800000 [ 9009.675085] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9010.603806] loop0: detected capacity change from 0 to 204800000 [ 9010.626116] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9013.905854] loop0: detected capacity change from 0 to 204800000 [ 9013.926916] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9017.114947] loop0: detected capacity change from 0 to 204800000 [ 9017.156845] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9020.410510] loop0: detected capacity change from 0 to 204800000 [ 9020.437319] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9023.156917] loop0: detected capacity change from 0 to 204800000 [ 9023.191896] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9023.228732] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9023.309300] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9023.323358] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9023.342393] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9023.360877] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9023.374774] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9023.387196] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9023.401238] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9023.415423] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9023.426765] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9023.442554] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9023.455921] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9023.469938] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9023.482276] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9023.495987] Lustre: 280135:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9028.625077] Lustre: server umount MGS complete [ 9029.317728] loop0: detected capacity change from 0 to 204800000 [ 9029.375426] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9029.706354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9029.751747] Lustre: lustre-MDT0000: Aborting client recovery [ 9030.621866] loop1: detected capacity change from 0 to 204800000 [ 9030.721706] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9030.840203] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9030.852624] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9030.861350] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9030.874990] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9030.886086] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9030.893271] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9030.900955] Lustre: 280566:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9030.917784] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9031.096753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9031.131475] Lustre: lustre-MDT0001: Aborting client recovery [ 9031.925362] loop2: detected capacity change from 0 to 204800000 [ 9032.052677] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9032.124927] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9032.133923] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9032.140562] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9032.148934] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9032.156469] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9032.168818] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9032.176921] Lustre: 280645:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9032.280672] Lustre: Found index 0 for lustre-OST0000, updating log [ 9032.318571] Lustre: 280664:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9032.376093] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9032.395205] Lustre: lustre-OST0000: Aborting recovery [ 9033.107220] loop3: detected capacity change from 0 to 204800000 [ 9033.189112] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9033.231173] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9033.245075] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9033.255096] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9033.265358] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9033.282757] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9033.298454] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9033.308819] Lustre: 280729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9033.329354] Lustre: Found index 1 for lustre-OST0001, updating log [ 9033.371746] Lustre: 280734:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9033.472590] Lustre: lustre-OST0001: Aborting recovery [ 9038.863655] LustreError: 280509:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 9039.846746] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9039.866200] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9039.878603] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9039.900174] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9040.872150] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9040.876453] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9046.072982] LustreError: 280476:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1887us: rc = -2 [ 9046.586253] LustreError: 280475:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2619us: rc = -2 [ 9046.612062] LustreError: 280475:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [ 9061.348041] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9061.348360] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9061.360522] Lustre: Skipped 3 previous similar messages [ 9061.371979] Lustre: Skipped 3 previous similar messages [ 9064.908543] Lustre: server umount lustre-MDT0000 complete [ 9065.702243] LustreError: 280460:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788849941 with bad export cookie 18364114046424423671 [ 9065.713297] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9066.465629] LustreError: 280961:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9066.469413] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9066.472807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9066.482816] LustreError: 280961:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9066.503304] Lustre: Skipped 1 previous similar message [ 9070.496856] LustreError: 280473:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9070.496888] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9070.510686] LustreError: 280473:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9070.521592] Lustre: Skipped 2 previous similar messages [ 9072.083699] Lustre: server umount lustre-MDT0001 complete [ 9078.969134] Lustre: server umount lustre-OST0000 complete [ 9085.897922] Lustre: server umount lustre-OST0001 complete [ 9092.569526] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:46:06 (1788849966) [ 9112.854678] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 9120.110051] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 9144.638666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9144.802512] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9144.818105] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9144.874641] Lustre: lustre-MDT0000: new disk, initializing [ 9144.916733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9144.922752] Lustre: Skipped 1 previous similar message [ 9144.934098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9147.532562] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9155.304417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9155.373311] Lustre: 284793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9155.419647] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9155.423171] Lustre: Skipped 1 previous similar message [ 9155.500715] Lustre: lustre-MDT0001: new disk, initializing [ 9155.594526] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9155.630071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9155.644471] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9158.540621] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9164.833801] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9168.131348] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9172.136978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9172.389270] Lustre: lustre-OST0000: new disk, initializing [ 9172.395123] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9172.401166] Lustre: 286760:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9172.488348] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9176.605375] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9181.207229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9181.215590] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9181.279300] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9183.012150] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9186.660782] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9193.448575] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9201.047808] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9242.594278] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9242.598166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9242.601063] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9242.608448] Lustre: Skipped 1 previous similar message [ 9242.616192] Lustre: Skipped 1 previous similar message [ 9247.720700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9247.728746] Lustre: Skipped 1 previous similar message [ 9255.392150] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9255.480861] Lustre: server umount lustre-OST0000 complete [ 9257.953963] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9257.955346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9257.956407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9257.956412] Lustre: Skipped 2 previous similar messages [ 9257.964881] Lustre: Skipped 1 previous similar message [ 9257.968601] LustreError: Skipped 1 previous similar message [ 9263.817659] Lustre: server umount lustre-MDT0000 complete [ 9266.047474] LustreError: 286759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850141 with bad export cookie 18364114046424426387 [ 9266.051265] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9266.057893] LustreError: 286759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 9266.264218] Lustre: server umount lustre-MDT0001 complete [ 9271.320030] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9273.895652] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9276.854825] Lustre: server umount MGS complete [ 9283.714929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9283.990672] LustreError: 290670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9284.004067] LustreError: 290683:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9284.047212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9286.428382] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9289.187441] LustreError: 290671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9289.206746] LustreError: 290671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9291.708406] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9295.009592] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9299.682272] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9303.132396] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9306.937202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9310.285191] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9314.305517] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9326.562804] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9326.572876] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9326.574136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9326.581580] Lustre: Skipped 4 previous similar messages [ 9331.874628] Lustre: server umount lustre-OST0000 complete [ 9336.801934] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9336.810735] Lustre: Skipped 1 previous similar message [ 9340.160479] Lustre: server umount lustre-MDT0000 complete [ 9341.921167] LustreError: 290665:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9341.934980] LustreError: 290665:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9341.998357] LustreError: 290652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850217 with bad export cookie 18364114046424428214 [ 9342.002296] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9342.003517] LustreError: 290652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9342.160986] Lustre: server umount lustre-MDT0001 complete [ 9347.257890] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 9348.825924] Key type lgssc unregistered [ 9349.014828] LNet: 294606:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9349.020162] LNetError: 294606:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9349.029701] LNet: Removed LNI 192.168.203.111@tcp [ 9349.523266] Key type .llcrypt unregistered [ 9349.525541] Key type ._llcrypt unregistered [ 9369.469432] Key type ._llcrypt registered [ 9369.472794] Key type .llcrypt registered [ 9369.524056] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 9378.069165] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 9378.602209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9378.646322] alg: No test for adler32 (adler32-zlib) [ 9379.558591] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 9379.714928] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 9381.346301] Key type lgssc registered [ 9381.896751] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9400.985554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9400.997132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9402.104330] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9402.120345] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9402.162560] Lustre: lustre-MDT0000: new disk, initializing [ 9402.196123] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9402.204334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9404.149234] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9409.818452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9409.874304] Lustre: 298531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9409.909449] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9409.912091] Lustre: Skipped 1 previous similar message [ 9409.961062] Lustre: lustre-MDT0001: new disk, initializing [ 9410.016229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9410.033903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9410.042614] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9411.967481] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9417.008116] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9419.625397] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9422.832763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9422.961800] Lustre: lustre-OST0000: new disk, initializing [ 9422.965386] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9422.970817] Lustre: 300498:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9423.027686] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9426.014513] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9430.914849] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9431.046547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9431.051050] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9431.088215] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9435.073489] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9439.567914] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9446.388390] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9456.499472] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9497.575754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9497.575955] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9497.578561] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9497.586235] Lustre: Skipped 1 previous similar message [ 9497.598898] LustreError: Skipped 1 previous similar message [ 9499.510164] Lustre: server umount lustre-OST0000 complete [ 9502.177268] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9502.181749] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9502.189527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9502.193334] Lustre: Skipped 1 previous similar message [ 9507.753924] Lustre: server umount lustre-MDT0000 complete [ 9509.331284] LustreError: 301353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850385 with bad export cookie 14564447946462054365 [ 9509.334076] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9509.337436] LustreError: 301353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9509.464609] Lustre: server umount lustre-MDT0001 complete [ 9513.151373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9514.952674] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9517.189895] Lustre: server umount MGS complete [ 9522.244316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9522.453945] LustreError: 304366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9522.463316] LustreError: 304366:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9522.465906] LustreError: 304379:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9522.501113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9524.233631] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9527.778202] LustreError: 304367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9528.501269] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9528.695092] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9530.572263] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9534.700067] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9537.402303] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9540.927398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9541.109587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9544.020257] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9547.995990] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9567.712992] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9567.717730] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9567.726274] Lustre: Skipped 1 previous similar message [ 9567.728975] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9567.732167] Lustre: Skipped 1 previous similar message [ 9573.216611] Lustre: server umount lustre-OST0000 complete [ 9578.977629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9578.978506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9578.984099] Lustre: Skipped 2 previous similar messages [ 9578.987075] Lustre: Skipped 2 previous similar messages [ 9584.097549] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9584.101451] Lustre: Skipped 2 previous similar messages [ 9589.216176] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9589.297286] Lustre: server umount lustre-MDT0000 complete [ 9590.864551] LustreError: 307090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788850466 with bad export cookie 14564447946462056290 [ 9590.866050] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9590.871650] LustreError: 307090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9590.997433] Lustre: server umount lustre-MDT0001 complete [ 9595.616921] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [ 9596.877580] Key type lgssc unregistered [ 9597.019514] LNet: 308307:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9597.024907] LNetError: 308307:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9597.038898] LNet: Removed LNI 192.168.203.111@tcp [ 9597.402465] Key type .llcrypt unregistered [ 9597.404165] Key type ._llcrypt unregistered [ 9606.232838] Key type ._llcrypt registered [ 9606.234190] Key type .llcrypt registered [ 9606.282337] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [ 9613.489074] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 9613.931435] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9613.964488] alg: No test for adler32 (adler32-zlib) [ 9614.848511] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [ 9614.963837] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [ 9616.568136] Key type lgssc registered [ 9617.060049] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9627.399635] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [ 9633.754995] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9633.766877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9634.868913] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9634.883130] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9634.923936] Lustre: lustre-MDT0000: new disk, initializing [ 9634.966692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9634.976192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9636.649055] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [ 9641.523518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9641.645739] Lustre: lustre-OST0000: new disk, initializing [ 9641.648486] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9641.651603] Lustre: Skipped 1 previous similar message [ 9641.655195] Lustre: 312329:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9641.688140] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9641.815986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9641.820642] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9641.834713] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9644.088779] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10220.148115] Lustre: server umount lustre-MDT0000 complete [10224.967066] LustreError: 312328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851100 with bad export cookie 4449264112330547247 [10224.981627] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10234.565831] Lustre: server umount lustre-OST0000 complete [10264.421403] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [10276.141384] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [10316.053932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10316.260538] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10316.304692] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10316.417267] Lustre: lustre-MDT0000: new disk, initializing [10316.557982] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10316.574721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10321.098628] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10331.916041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10332.097913] Lustre: 316633:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10332.158789] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10332.164193] Lustre: Skipped 1 previous similar message [10332.246436] Lustre: lustre-MDT0001: new disk, initializing [10332.382293] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10332.440882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10332.458159] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10337.072281] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10347.271630] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10353.350288] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10361.035834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10361.348197] Lustre: lustre-OST0000: new disk, initializing [10361.355440] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10361.368968] Lustre: 318602:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10361.491353] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10367.073686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10367.093623] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10367.283164] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10368.010817] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10378.408129] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10383.402363] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10383.618363] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10388.035551] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10388.250672] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10393.060068] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10393.060357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10393.072176] LustreError: Skipped 1 previous similar message [10393.087755] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10398.180522] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10398.186854] Lustre: Skipped 2 previous similar messages [10403.300170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10403.305478] Lustre: Skipped 1 previous similar message [10404.320254] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10404.481445] Lustre: server umount lustre-OST0000 complete [10408.420410] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10408.451586] Lustre: Skipped 1 previous similar message [10408.458200] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10413.538151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10413.544517] Lustre: Skipped 2 previous similar messages [10414.604698] Lustre: server umount lustre-MDT0000 complete [10417.988373] LustreError: 318601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851293 with bad export cookie 4449264112333488024 [10417.995055] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10418.005090] LustreError: 318601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10418.247521] Lustre: server umount lustre-MDT0001 complete [10421.465326] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10423.686382] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:08:17 (1788851297) [10430.982346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10431.539704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10435.380069] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10436.581731] LustreError: 320916:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10436.609317] LustreError: 320916:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10441.701704] LustreError: 320917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10443.738390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10444.061326] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10448.867114] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10457.503165] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10464.747876] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10472.606345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10473.023888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10479.571232] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10486.865031] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10493.042625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10493.211270] Lustre: lustre-OST0001: new disk, initializing [10493.215424] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10493.222738] Lustre: 323755:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10493.324373] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10499.467499] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10500.232975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10500.241163] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10500.277486] LustreError: 324153:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10501.346718] LustreError: 324153:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10501.360300] LustreError: 324153:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10502.370165] LustreError: 324167:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10504.419467] LustreError: 324167:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10504.437030] LustreError: 324167:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10508.583365] LustreError: 324153:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10508.592150] LustreError: 324153:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10509.414455] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10509.699161] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10515.380467] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9612fc633000.ost_server_uuid 50 [10516.639597] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9612fc633000.ost_server_uuid in FULL state after 0 sec [10516.775110] LustreError: 324167:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10516.782381] LustreError: 324167:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10521.551303] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9612fc633000.ost_server_uuid 50 [10523.222351] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9612fc633000.ost_server_uuid in FULL state after 0 sec [10524.982347] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10540.518349] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10540.531481] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10540.546686] Lustre: Skipped 1 previous similar message [10540.554499] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10540.574381] Lustre: Skipped 1 previous similar message [10544.937557] Lustre: server umount lustre-OST0001 complete [10545.714592] LustreError: 322739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10551.779856] LustreError: 322739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10551.810273] LustreError: 322739:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10553.314915] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10553.323321] LustreError: Skipped 1 previous similar message [10553.326531] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10553.337899] Lustre: Skipped 1 previous similar message [10555.366721] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10556.902653] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10556.909257] LustreError: 323464:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10556.913342] Lustre: Skipped 6 previous similar messages [10556.941220] LustreError: 323464:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10558.201532] Lustre: server umount lustre-OST0000 complete [10562.018512] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10562.022566] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10562.026287] Lustre: Skipped 1 previous similar message [10567.978294] Lustre: server umount lustre-MDT0000 complete [10571.099378] LustreError: 320898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851446 with bad export cookie 4449264112333488906 [10571.117591] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10571.391497] Lustre: server umount lustre-MDT0001 complete [10582.505955] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [10584.954426] Key type lgssc unregistered [10585.277615] LNet: 326439:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10585.290554] LNetError: 326439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10585.320028] LNet: Removed LNI 192.168.203.111@tcp [10586.278215] Key type .llcrypt unregistered [10586.280817] Key type ._llcrypt unregistered [10613.936322] Key type ._llcrypt registered [10613.941503] Key type .llcrypt registered [10614.090859] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [10615.125797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10615.196438] alg: No test for adler32 (adler32-zlib) [10616.322721] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [10616.566994] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [10618.268456] Key type lgssc registered [10619.346383] Lustre: Echo OBD driver; http://www.lustre.org/ [10626.681960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10626.710210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10628.181932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10632.131454] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10638.309586] LustreError: 327860:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10638.325083] LustreError: 327860:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10638.639393] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10643.426931] LustreError: 327861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10644.671045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10644.924697] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10648.834172] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10658.561217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10658.923086] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10667.106852] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10669.178907] LustreError: 329541:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10673.329535] LustreError: 329542:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10673.357628] LustreError: 329542:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10673.359945] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10677.037389] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10678.771234] LustreError: 329985:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10678.807025] LustreError: 329985:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10685.711922] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10685.952532] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10691.106479] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10694.077679] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10704.825984] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10712.883682] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10713.011353] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10824.164738] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10824.168024] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10824.195375] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10824.205353] LustreError: Skipped 1 previous similar message [10824.218523] Lustre: Skipped 1 previous similar message [10825.842549] Lustre: server umount lustre-OST0001 complete [10829.295318] LustreError: 330557:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10829.321415] LustreError: 330557:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10832.361074] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10832.370739] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10832.386803] Lustre: Skipped 1 previous similar message [10832.397431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10833.380029] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10833.394280] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10833.413094] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10837.440235] Lustre: server umount lustre-OST0000 complete [10844.643023] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10844.644641] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10844.645209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10844.645214] Lustre: Skipped 2 previous similar messages [10844.653797] Lustre: Skipped 1 previous similar message [10846.919386] Lustre: server umount lustre-MDT0000 complete [10849.763171] LustreError: 327854:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10849.777114] LustreError: 327854:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10849.911118] LustreError: 327841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851725 with bad export cookie 3239733732636513337 [10849.914802] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10849.921081] LustreError: 327841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10850.151992] Lustre: server umount lustre-MDT0001 complete [10859.038309] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [10861.323547] Key type lgssc unregistered [10861.588751] LNet: 332807:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10861.602785] LNetError: 332807:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10861.618652] LNet: Removed LNI 192.168.203.111@tcp [10862.444610] Key type .llcrypt unregistered [10862.446708] Key type ._llcrypt unregistered [10904.186527] Key type ._llcrypt registered [10904.193066] Key type .llcrypt registered [10904.337778] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [10905.550072] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10905.582968] alg: No test for adler32 (adler32-zlib) [10906.670901] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [10906.926779] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [10908.672258] Key type lgssc registered [10909.786077] Lustre: Echo OBD driver; http://www.lustre.org/ [10918.947939] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10918.962481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10920.388055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10924.646260] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10930.660210] LustreError: 335091:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10930.670228] LustreError: 335091:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10934.129081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10934.663722] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10939.553407] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10942.522606] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10949.012492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10949.487398] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10955.730642] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10959.721434] LustreError: 336580:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10963.905866] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10964.180248] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10965.247300] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10965.260594] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10970.680951] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [10979.495356] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10985.544246] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10996.208102] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10996.212579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10996.235452] Lustre: Skipped 2 previous similar messages [10996.243906] Lustre: Skipped 3 previous similar messages [10996.256164] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11001.323974] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11001.330669] Lustre: Skipped 3 previous similar messages [11001.551786] Lustre: server umount lustre-MDT0000 complete [11004.685934] LustreError: 335072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788851880 with bad export cookie 13231740294632102536 [11004.690595] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11004.703912] LustreError: 335072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11005.077766] Lustre: server umount lustre-MDT0001 complete [11018.318983] Lustre: server umount lustre-OST0000 complete [11033.166283] Lustre: server umount lustre-OST0001 complete [11048.425121] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [11061.808906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11062.497349] LustreError: 340356:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11062.671473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11067.879343] LustreError: 340357:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11068.550603] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11073.008288] LustreError: 340357:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11079.094948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11079.748547] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11085.018406] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11087.704135] Lustre: 341498:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11094.034455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11100.685113] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11104.551236] LustreError: 341853:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11104.587391] LustreError: 341853:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11107.640033] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11110.593283] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11110.910553] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11110.920286] Lustre: Skipped 1 previous similar message [11116.049575] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11120.863676] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11131.289321] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11134.281946] Lustre: 343370:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11141.090311] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11141.100777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11141.113691] Lustre: Skipped 1 previous similar message [11141.117470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11143.453921] Lustre: server umount lustre-MDT0000 complete [11146.723688] LustreError: 340357:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11146.764224] LustreError: 340357:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11146.802692] LustreError: 343372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788852022 with bad export cookie 13231740294632104237 [11146.814739] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11146.820034] LustreError: 343372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11147.204664] Lustre: server umount lustre-MDT0001 complete [11161.181327] Lustre: server umount lustre-OST0000 complete [11175.197429] Lustre: server umount lustre-OST0001 complete [11179.416923] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11181.614777] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11183.931808] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:20:57 (1788852057) [11207.975335] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [11216.384196] loop0: detected capacity change from 0 to 1099511627776 [11216.817650] SGI XFS with ACLs, security attributes, quota, no debug enabled [11216.822163] *** VALIDATE xfs *** [11216.830260] XFS (loop0): Mounting V5 Filesystem [11216.850397] XFS (loop0): Ending clean mount [11218.308314] loop1: detected capacity change from 0 to 18691697672192 [11311.655141] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [11321.205202] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [11358.555338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11358.890283] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11358.922951] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11359.060770] Lustre: lustre-MDT0000: new disk, initializing [11359.185890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11359.220141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11363.594637] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11376.771084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11377.037056] Lustre: 348506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11377.119195] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11377.137198] Lustre: Skipped 1 previous similar message [11377.238215] Lustre: lustre-MDT0001: new disk, initializing [11377.434184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11377.445286] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11383.001831] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11393.679163] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11399.668538] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11407.405820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11407.795430] Lustre: lustre-OST0000: new disk, initializing [11407.802409] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11407.822826] Lustre: 350474:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11409.529176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11409.539558] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11409.626757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11414.960418] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11428.492839] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11435.919235] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11436.206300] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11443.524514] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11443.649202] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11445.739961] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11445.740891] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11445.752886] LustreError: Skipped 1 previous similar message [11445.764755] Lustre: Skipped 3 previous similar messages [11445.767117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11445.767126] Lustre: Skipped 3 previous similar messages [11450.856317] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11450.866369] Lustre: Skipped 2 previous similar messages [11451.639658] Lustre: server umount lustre-OST0000 complete [11459.556960] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11459.564753] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11459.589254] Lustre: Skipped 1 previous similar message [11459.598346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11462.433600] Lustre: server umount lustre-MDT0000 complete [11465.922593] LustreError: 348497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788852341 with bad export cookie 13231740294632105784 [11465.925756] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11465.931652] LustreError: 348497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11466.226951] Lustre: server umount lustre-MDT0001 complete [11476.071280] XFS (loop0): Unmounting Filesystem [11483.384527] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:25:56 (1788852356) [11491.552343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11491.954898] LustreError: 353035:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11492.007110] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11492.012469] Lustre: Skipped 2 previous similar messages [11496.543392] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11505.615208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11510.553194] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11517.995224] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11523.224935] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11529.445741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11535.973803] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11543.930523] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11551.713975] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11551.723948] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11551.754621] Lustre: Skipped 1 previous similar message [11551.774426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11551.780961] Lustre: Skipped 2 previous similar messages [11557.866528] LustreError: 354857:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11557.905231] LustreError: 354857:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11558.095757] Lustre: server umount lustre-OST0000 complete [11562.466946] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11562.473221] LustreError: Skipped 1 previous similar message [11562.477454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11562.493086] Lustre: Skipped 1 previous similar message [11576.290155] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11576.580363] Lustre: server umount lustre-MDT0000 complete [11581.469806] LustreError: 353017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788852457 with bad export cookie 13231740294632106666 [11581.474530] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11581.483671] LustreError: 353017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11581.815238] Lustre: server umount lustre-MDT0001 complete [11591.837282] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [11594.565282] Key type lgssc unregistered [11594.952552] LNet: 356662:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11594.963792] LNetError: 356662:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11594.976266] LNet: Removed LNI 192.168.203.111@tcp [11595.957146] Key type .llcrypt unregistered [11595.962718] Key type ._llcrypt unregistered [11626.183940] Key type ._llcrypt registered [11626.186348] Key type .llcrypt registered [11626.288164] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [11626.779604] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11626.819259] alg: No test for adler32 (adler32-zlib) [11627.864737] Lustre: Lustre: Build Version: 2.17.57_114_gc92b566 [11628.079897] LNet: Added LNI 192.168.203.111@tcp [8/256/0/180] [11629.728207] Key type lgssc registered [11630.688616] Lustre: Echo OBD driver; http://www.lustre.org/ [11638.788535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11638.803866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11640.406055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11644.603796] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11650.537340] LustreError: 358229:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11650.559816] LustreError: 358229:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11654.132605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11654.566093] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11659.454374] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11669.185736] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11679.815878] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11692.500456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11693.026423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11701.748420] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11710.473570] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11720.697189] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11720.701862] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11720.728326] Lustre: Skipped 1 previous similar message [11720.759023] Lustre: Skipped 1 previous similar message [11723.340111] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.11@tcp (stopping) [11725.274095] Lustre: server umount lustre-MDT0001 complete [11725.802904] LustreError: 358958:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11725.852408] LustreError: 358958:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11728.429940] LustreError: 358225:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11730.922949] LustreError: 360543:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11730.945892] LustreError: 360543:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11736.034580] LustreError: 358958:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11736.072433] LustreError: 358958:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11738.785092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11738.927816] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11738.932094] Lustre: Found index 1 for lustre-MDT0001, updating log [11739.004861] Lustre: 358219:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11739.321306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11743.347186] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [11744.762995] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11746.819944] LustreError: 361833:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11746.830125] LustreError: 361833:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11746.834757] LustreError: 358221:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11746.867528] LustreError: 361836:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.111@tcp: cfg command failed: rc = -17 [11746.872365] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11746.872365] [12051.942610] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12051.958800] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12051.966728] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12056.158746] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.11@tcp (stopping) [12056.167478] Lustre: Skipped 1 previous similar message [12057.923225] Lustre: server umount lustre-MDT0001 complete [12061.228357] LustreError: 358224:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12061.260260] LustreError: 358224:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12069.089967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12069.164280] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12069.167067] Lustre: Found index 1 for lustre-MDT0001, updating log [12069.198174] Lustre: 358216:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12069.452000] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12074.303144] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [12074.471886] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12075.505729] LustreError: 370318:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12075.513205] LustreError: 370318:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12075.520185] LustreError: 358221:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12075.526692] LustreError: 358221:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12075.577820] LustreError: 370327:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12075.594180] LustreError: 370327:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12075.611229] LustreError: 370327:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.111@tcp: cfg command failed: rc = -17 [12075.631864] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.111@tcp [12075.631864] [12381.666835] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12381.670426] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12381.712237] Lustre: Skipped 1 previous similar message [12381.716353] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12381.754758] Lustre: Skipped 3 previous similar messages [12385.816940] Lustre: server umount lustre-MDT0001 complete [12386.796588] LustreError: 358958:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12386.828452] LustreError: 358958:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [12400.116818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12400.351708] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12400.370171] Lustre: Found index 1 for lustre-MDT0001, updating log [12400.440337] Lustre: 358216:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12401.022931] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12406.104836] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [12406.260306] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12409.361448] LustreError: 378929:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12409.375052] LustreError: 378929:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12409.392486] LustreError: 358221:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12409.395387] LustreError: 358221:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12409.468979] LustreError: 378932:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12409.507677] LustreError: 378932:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.203.111@tcp: cfg command failed: rc = -114 [12409.526119] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12409.526119] [12728.814099] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12728.820161] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12728.836861] Lustre: Skipped 1 previous similar message [12728.852734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12728.860408] Lustre: Skipped 1 previous similar message [12734.837256] Lustre: server umount lustre-MDT0000 complete [12736.997653] LustreError: 358226:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12737.029766] LustreError: 358226:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [12739.229373] LustreError: 358210:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788853615 with bad export cookie 10539541303700187403 [12739.230839] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12739.238335] LustreError: 358210:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12742.116479] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12742.141518] Lustre: Skipped 2 previous similar messages [12746.200280] Lustre: server umount lustre-MDT0001 complete [12753.471984] Lustre: server umount lustre-OST0000 complete [12765.745420] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [12780.037996] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [12820.376996] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:48:13 (1788853693) [12827.658121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12827.935264] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12827.959795] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12828.034395] Lustre: lustre-MDT0000: new disk, initializing [12828.081257] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12828.099192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12833.445967] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [12845.098239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12845.263113] Lustre: 389434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12845.309451] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12845.313600] Lustre: Skipped 1 previous similar message [12845.377301] Lustre: lustre-MDT0001: new disk, initializing [12845.452248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12845.495629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12845.509654] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12850.717291] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [12861.530178] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12869.834413] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12879.250489] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12879.595871] Lustre: lustre-OST0000: new disk, initializing [12879.600435] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12879.608985] Lustre: 391404:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12879.734652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12887.123952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12887.136394] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12887.274905] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12888.928346] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_default_debug -1 all [12904.415936] Lustre: DEBUG MARKER: oleg311-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12915.177329] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12915.186687] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12915.202855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12915.205772] Lustre: Skipped 5 previous similar messages [12916.208033] Lustre: server umount lustre-MDT0000 complete [12918.249525] LustreError: 390485:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12918.285621] LustreError: 390485:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12920.476419] LustreError: 389426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788853796 with bad export cookie 10539541303700189356 [12920.491136] LustreError: MGC192.168.203.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12920.992582] Lustre: server umount lustre-MDT0001 complete [12936.052521] Lustre: server umount lustre-OST0000 complete [12973.719763] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing set_hostid [12983.948717] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing load_modules_local [13019.397759] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12753 sec ========= 03:51:32 (1788853892) [13021.818445] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:51:34 (1788853894) === [13026.584160] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:51:39 (1788853899) === [13066.249526] Lustre: DEBUG MARKER: oleg311-server.virtnet: executing unload_modules_local [13069.605757] Key type lgssc unregistered [13069.869536] LNet: 397772:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13069.878272] LNetError: 397772:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13069.909439] LNet: Removed LNI 192.168.203.111@tcp [13071.034839] Key type .llcrypt unregistered [13071.038153] Key type ._llcrypt unregistered