[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 500720324 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003200] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.007672] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010011] pid_max: default: 32768 minimum: 301 [ 0.011133] LSM: Security Framework initializing [ 0.012052] Yama: becoming mindful. [ 0.013049] SELinux: Initializing. [ 0.014080] *** VALIDATE selinux *** [ 0.022652] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027393] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028146] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029111] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030119] *** VALIDATE tmpfs *** [ 0.032137] *** VALIDATE proc *** [ 0.033224] *** VALIDATE cgroup *** [ 0.034009] *** VALIDATE cgroup2 *** [ 0.036131] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037147] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040014] Spectre V2 : User space: Vulnerable [ 0.041009] Speculative Store Bypass: Vulnerable [ 0.043961] debug: unmapping init [mem 0xffffffff95259000-0xffffffff95260fff] [ 0.045152] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046702] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047028] ... version: 2 [ 0.048013] ... bit width: 48 [ 0.049015] ... generic registers: 4 [ 0.050013] ... value mask: 0000ffffffffffff [ 0.051015] ... max period: 00007fffffffffff [ 0.052016] ... fixed-purpose events: 3 [ 0.053014] ... event mask: 000000070000000f [ 0.054296] rcu: Hierarchical SRCU implementation. [ 0.056442] smp: Bringing up secondary CPUs ... [ 0.057778] x86: Booting SMP configuration: [ 0.058024] .... node #0, CPUs: #1 #2 #3 [ 0.064019] smp: Brought up 1 node, 4 CPUs [ 0.066016] smpboot: Max logical packages: 1 [ 0.067013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.168172] node 0 deferred pages initialised in 98ms [ 0.173252] devtmpfs: initialized [ 0.176387] x86/mm: Memory block size: 128MB [ 0.180285] gcov: version magic: 0x41383552 [ 0.183374] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.188267] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.191564] pinctrl core: initialized pinctrl subsystem [ 0.194392] [ 0.195011] ************************************************************* [ 0.197020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.200023] ** ** [ 0.203018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.206017] ** ** [ 0.208021] ** This means that this kernel is built to expose internal ** [ 0.211036] ** IOMMU data structures, which may compromise security on ** [ 0.214021] ** your system. ** [ 0.217020] ** ** [ 0.220021] ** If you see this message and you are not debugging the ** [ 0.222019] ** kernel, report this immediately to your vendor! ** [ 0.225021] ** ** [ 0.227026] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.230024] ************************************************************* [ 0.234717] NET: Registered protocol family 16 [ 0.236642] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.240090] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.243098] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.247153] cpuidle: using governor menu [ 0.248828] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.253548] PCI: Using configuration type 1 for base access [ 0.256163] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.265223] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.266000] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.269488] cryptd: max_cpu_qlen set to 1000 [ 0.271217] ACPI: Added _OSI(Module Device) [ 0.272049] ACPI: Added _OSI(Processor Device) [ 0.274024] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.275064] ACPI: Added _OSI(Processor Aggregator Device) [ 0.282109] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.291593] ACPI: Interpreter enabled [ 0.293184] ACPI: PM: (supports S0 S3 S4 S5) [ 0.296019] ACPI: Using IOAPIC for interrupt routing [ 0.298182] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.307605] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.321483] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.324071] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.328028] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.332110] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.337056] acpiphp: Slot [2] registered [ 0.339319] acpiphp: Slot [5] registered [ 0.341274] acpiphp: Slot [6] registered [ 0.343223] acpiphp: Slot [7] registered [ 0.345223] acpiphp: Slot [8] registered [ 0.346210] acpiphp: Slot [9] registered [ 0.348208] acpiphp: Slot [10] registered [ 0.350249] acpiphp: Slot [3] registered [ 0.352143] acpiphp: Slot [4] registered [ 0.354189] acpiphp: Slot [11] registered [ 0.355225] acpiphp: Slot [12] registered [ 0.357160] acpiphp: Slot [13] registered [ 0.359168] acpiphp: Slot [14] registered [ 0.361152] acpiphp: Slot [15] registered [ 0.363230] acpiphp: Slot [16] registered [ 0.365169] acpiphp: Slot [17] registered [ 0.367141] acpiphp: Slot [18] registered [ 0.368157] acpiphp: Slot [19] registered [ 0.370213] acpiphp: Slot [20] registered [ 0.372163] acpiphp: Slot [21] registered [ 0.373215] acpiphp: Slot [22] registered [ 0.375201] acpiphp: Slot [23] registered [ 0.378253] acpiphp: Slot [24] registered [ 0.379282] acpiphp: Slot [25] registered [ 0.381373] acpiphp: Slot [26] registered [ 0.383261] acpiphp: Slot [27] registered [ 0.385354] acpiphp: Slot [28] registered [ 0.387149] acpiphp: Slot [29] registered [ 0.389148] acpiphp: Slot [30] registered [ 0.391161] acpiphp: Slot [31] registered [ 0.393120] PCI host bridge to bus 0000:00 [ 0.395031] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.398037] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.400030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.403034] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.405036] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.407037] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.409226] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.413347] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.417951] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.430898] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.436065] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.439026] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.442026] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.443027] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.446526] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.451102] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.453237] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.457706] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.464023] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.481036] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.488023] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.495037] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.507000] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.517029] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.550022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.568132] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.575021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.581022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.597025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.611152] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.620021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.634023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.664026] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.687068] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.701018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.719018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.743017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.759860] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.772029] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.783036] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.820026] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.843405] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.854038] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.865029] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.886024] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.900000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.902415] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.904519] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.908484] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.910215] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.914326] iommu: Default domain type: Passthrough [ 0.916290] SCSI subsystem initialized [ 0.917000] ACPI: bus type USB registered [ 0.918092] usbcore: registered new interface driver usbfs [ 0.919059] usbcore: registered new interface driver hub [ 0.921102] usbcore: registered new device driver usb [ 0.922190] pps_core: LinuxPPS API ver. 1 registered [ 0.923013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.927058] PTP clock support registered [ 0.929130] EDAC MC: Ver: 3.0.0 [ 0.931158] PCI: Using ACPI for IRQ routing [ 0.932896] NetLabel: Initializing [ 0.933012] NetLabel: domain hash size = 128 [ 0.935009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.936160] NetLabel: unlabeled traffic allowed by default [ 0.938220] vgaarb: loaded [ 0.939336] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.942024] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.951925] clocksource: Switched to clocksource kvm-clock [ 1.083612] VFS: Disk quotas dquot_6.6.0 [ 1.085241] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.086890] *** VALIDATE ramfs *** [ 1.087874] *** VALIDATE hugetlbfs *** [ 1.091215] pnp: PnP ACPI init [ 1.093420] pnp: PnP ACPI: found 6 devices [ 1.115343] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.119201] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.120642] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.122236] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.124380] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.126604] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.128136] NET: Registered protocol family 2 [ 1.130421] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.134929] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.137969] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.142922] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.145794] TCP: Hash tables configured (established 65536 bind 65536) [ 1.148240] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.151421] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.155586] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.160631] NET: Registered protocol family 1 [ 1.166656] RPC: Registered named UNIX socket transport module. [ 1.169965] RPC: Registered udp transport module. [ 1.171961] RPC: Registered tcp transport module. [ 1.173950] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.176690] NET: Registered protocol family 44 [ 1.178477] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.181115] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.184403] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.189455] PCI: CLS 0 bytes, default 64 [ 1.192767] Unpacking initramfs... [ 2.669812] debug: unmapping init [mem 0xffff891e7cc54000-0xffff891e7ffbffff] [ 2.674180] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.676213] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.679790] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.275160] Initialise system trusted keyrings [ 3.277014] Key type blacklist registered [ 3.278775] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.288399] zbud: loaded [ 3.291351] *** VALIDATE nfs *** [ 3.292643] *** VALIDATE nfs4 *** [ 3.294174] pstore: using deflate compression [ 3.298196] Platform Keyring initialized [ 3.415935] NET: Registered protocol family 38 [ 3.417565] Key type asymmetric registered [ 3.418641] Asymmetric key parser 'x509' registered [ 3.420345] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.423141] io scheduler mq-deadline registered [ 3.424326] io scheduler kyber registered [ 3.426196] io scheduler bfq registered [ 3.427758] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.430345] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.432750] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.435920] ACPI: Power Button [PWRF] [ 3.441366] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.448152] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.486568] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.498808] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.537090] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.578229] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.608548] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.614554] Non-volatile memory driver v1.3 [ 3.616402] Linux agpgart interface v0.103 [ 3.654981] virtio_blk virtio1: [vda] 146680 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.658670] vda: detected capacity change from 0 to 75100160 [ 3.681752] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.685014] vdb: detected capacity change from 0 to 1073741824 [ 3.707828] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.710584] vdc: detected capacity change from 0 to 2621440000 [ 3.731001] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.735025] vdd: detected capacity change from 0 to 2621440000 [ 3.752648] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.755230] vde: detected capacity change from 0 to 4294967296 [ 3.771914] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.774146] vdf: detected capacity change from 0 to 4294967296 [ 3.780694] libphy: Fixed MDIO Bus: probed [ 3.794879] usbcore: registered new interface driver usbserial_generic [ 3.797014] usbserial: USB Serial support registered for generic [ 3.799428] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.806310] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.808633] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.811439] mousedev: PS/2 mouse device common for all mice [ 3.819449] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.821791] rtc_cmos 00:05: RTC can wake from S4 [ 3.838558] rtc_cmos 00:05: registered as rtc0 [ 3.843636] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.850284] intel_pstate: CPU model not supported [ 3.853650] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.863798] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.865924] hid: raw HID events driver (C) Jiri Kosina [ 3.874877] usbcore: registered new interface driver usbhid [ 3.878982] usbhid: USB HID core driver [ 3.882206] drop_monitor: Initializing network drop monitor service [ 3.886776] Initializing XFRM netlink socket [ 3.890416] NET: Registered protocol family 10 [ 3.895257] Segment Routing with IPv6 [ 3.897901] NET: Registered protocol family 17 [ 3.900663] mpls_gso: MPLS GSO support [ 3.908877] RAS: Correctable Errors collector initialized. [ 3.913144] AVX version of gcm_enc/dec engaged. [ 3.916308] AES CTR mode by8 optimization enabled [ 4.011498] sched_clock: Marking stable (4011478907, 0)->(4984841606, -973362699) [ 4.015642] registered taskstats version 1 [ 4.018444] Loading compiled-in X.509 certificates [ 4.021227] zswap: loaded using pool lzo/zbud [ 4.050800] Key type big_key registered [ 4.065555] Key type encrypted registered [ 4.067621] ima: No TPM chip found, activating TPM-bypass! [ 4.069907] ima: Allocated hash algorithm: sha1 [ 4.073645] ima: No architecture policies found [ 4.075883] evm: Initialising EVM extended attributes: [ 4.078622] evm: security.selinux [ 4.079974] evm: security.ima [ 4.081246] evm: security.capability [ 4.082729] evm: HMAC attrs: 0x1 [ 4.085230] rtc_cmos 00:05: setting system clock to 2026-09-13 02:04:11 UTC (1789265051) [ 4.092742] debug: unmapping init [mem 0xffffffff96203000-0xffffffff963fffff] [ 4.096279] debug: unmapping init [mem 0xffffffff94f82000-0xffffffff95258fff] [ 4.105145] Write protecting the kernel read-only data: 28672k [ 4.109967] debug: unmapping init [mem 0xffffffff93603000-0xffffffff937fffff] [ 4.113349] debug: unmapping init [mem 0xffffffff93f14000-0xffffffff93ffffff] [ 4.149758] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.158622] systemd[1]: Detected virtualization kvm. [ 4.160785] systemd[1]: Detected architecture x86-64. [ 4.162821] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.195291] systemd[1]: No hostname configured. [ 4.196979] systemd[1]: Set hostname to . [ 4.200879] random: systemd: uninitialized urandom read (16 bytes read) [ 4.205442] systemd[1]: Initializing machine ID from random generator. [ 4.271909] random: ln: uninitialized urandom read (6 bytes read) [ 4.385852] random: systemd: uninitialized urandom read (16 bytes read) [ 4.387934] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.393952] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.401915] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. [ OK ] Reached target Paths. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.206182] device-mapper: uevent: version 1.0.3 [ 5.207988] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.103767] random: fast init done [ 6.207595] virtio_net virtio0 ens2: renamed from eth0 [ 6.292852] scsi host0: ata_piix [ 6.424214] scsi host1: ata_piix [ 6.428846] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.431983] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.443355] dracut-initqueue[586]: RTNETLINK answers: File exists [ 11.588131] random: crng init done [ 11.590310] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 12.934114] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 14.433700] printk: systemd: 26 output lines suppressed due to ratelimiting [ 14.833461] SELinux: Disabled at runtime. [ 14.910925] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 14.922615] systemd[1]: Detected virtualization kvm. [ 14.924975] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 15.683667] systemd[1]: initrd-switch-root.service: Succeeded. [ 15.695056] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 15.703753] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 15.710527] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 15.716695] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 15.729323] systemd[1]: Starting Journal Service... Starting Journal Service... [ 15.741418] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Initrd Root File System. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-getty.slice. [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... [FAILED[ 15.950283] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS ] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 16.564241] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 17.128747] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 17.210543] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 17.608434] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 17.702896] EDAC sbridge: Ver: 1.1.2 [ 20.519496] Key type dns_resolver registered [ 20.901285] NFS: Registering the id_resolver key type [ 20.903769] Key type id_resolver registered [ 20.905652] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg444-server login: [ 43.601006] hrtimer: interrupt took 6411988 ns [ 77.019171] libcfs: loading out-of-tree module taints kernel. [ 77.085594] Key type ._llcrypt registered [ 77.093362] Key type .llcrypt registered [ 77.222494] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 97.343540] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 98.738624] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 98.751947] alg: No test for adler32 (adler32-zlib) [ 100.323126] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 101.098979] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 102.815231] Key type lgssc registered [ 105.078807] Lustre: Echo OBD driver; http://www.lustre.org/ [ 123.159213] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 161.457424] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 174.329185] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 174.387597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 175.543741] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 175.590363] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 175.650202] Lustre: lustre-MDT0000: new disk, initializing [ 175.712070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 175.730954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 179.604195] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 192.816269] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 192.902502] Lustre: 6512:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 192.925316] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 192.933572] Lustre: Skipped 1 previous similar message [ 193.026524] Lustre: lustre-MDT0001: new disk, initializing [ 193.109557] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 193.144208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 193.155249] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 197.729668] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 202.615065] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 212.561691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 212.878772] Lustre: lustre-OST0000: new disk, initializing [ 212.885161] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 212.894600] Lustre: 8452:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 212.965552] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 218.720294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 218.745412] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 218.853380] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 219.604545] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.061202] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 236.234212] Lustre: lustre-OST0001: new disk, initializing [ 236.239936] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 236.252400] Lustre: 9525:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 236.356525] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 243.841172] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 246.270956] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 246.276623] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 246.308669] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 254.792820] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 258.826573] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 264.864718] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing check_logdir /tmp/testlogs/ [ 268.466107] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing yml_node [ 272.851934] Lustre: DEBUG MARKER: Client: 2.17.57.106 [ 275.864337] Lustre: DEBUG MARKER: MDS: 2.17.57.106 [ 278.897697] Lustre: DEBUG MARKER: OSS: 2.17.57.106 [ 280.609764] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Sep 12 22:08:47 EDT 2026 [ 298.380312] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 299.901619] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 315.874318] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 315.879478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.885147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.920177] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 317.924064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 317.935418] Lustre: Skipped 2 previous similar messages [ 317.944814] Lustre: Skipped 3 previous similar messages [ 321.188521] Lustre: server umount lustre-MDT0000 complete [ 324.456837] LustreError: 9524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265371 with bad export cookie 14517709543151526836 [ 324.467177] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 328.167190] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 328.169088] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 328.180840] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 328.190203] Lustre: Skipped 1 previous similar message [ 328.236134] LustreError: 6521:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 329.184473] LustreError: 6519:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 329.221374] LustreError: 6519:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 330.920571] Lustre: server umount lustre-MDT0001 complete [ 334.460424] Lustre: server umount lustre-OST0000 complete [ 338.603556] Lustre: server umount lustre-OST0001 complete [ 345.044454] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 354.381446] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 388.568811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 388.809241] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 388.853176] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 388.916743] Lustre: lustre-MDT0000: new disk, initializing [ 388.985207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 388.998103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 393.479424] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 402.909351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 403.053867] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 403.130896] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 403.134281] Lustre: Skipped 1 previous similar message [ 403.250509] Lustre: lustre-MDT0001: new disk, initializing [ 403.423827] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 403.472764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 403.487620] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 407.498791] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 417.890464] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 425.029559] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 431.984233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 432.313566] Lustre: lustre-OST0000: new disk, initializing [ 432.328241] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 432.337333] Lustre: 17041:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 432.459511] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 434.271502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 434.283615] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 434.465652] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 439.700434] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 449.134459] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 453.835214] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 454.034971] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 458.150731] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 458.317686] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 460.261863] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 460.263355] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 460.271072] LustreError: Skipped 1 previous similar message [ 460.285834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 460.295603] Lustre: Skipped 4 previous similar messages [ 465.381038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 465.389105] Lustre: Skipped 1 previous similar message [ 474.079190] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 474.284795] Lustre: server umount lustre-OST0000 complete [ 480.224459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 480.231719] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 480.257651] Lustre: Skipped 1 previous similar message [ 480.260625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 480.271054] Lustre: Skipped 2 previous similar messages [ 483.962225] Lustre: server umount lustre-MDT0000 complete [ 485.856961] LustreError: 15951:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 485.882271] LustreError: 15951:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 487.127353] LustreError: 15065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265534 with bad export cookie 14517709543151528369 [ 487.130414] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 487.135840] LustreError: 15065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 487.518409] Lustre: server umount lustre-MDT0001 complete [ 492.319630] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 22:12:19 (1789265539) [ 498.405528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 498.745026] LustreError: 19309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 498.816851] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 503.621681] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 504.296597] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 512.527429] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.509673] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 524.191633] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 528.711299] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 535.119685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 542.512201] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 552.714496] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 556.707858] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 564.196040] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 564.202781] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 564.212219] Lustre: Skipped 1 previous similar message [ 564.216458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 564.219338] Lustre: Skipped 1 previous similar message [ 568.809265] Lustre: server umount lustre-MDT0000 complete [ 569.826391] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.845675] LustreError: 19310:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 572.424593] LustreError: 19290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265619 with bad export cookie 14517709543151529251 [ 572.435483] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 572.438748] LustreError: 19290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 572.788742] Lustre: server umount lustre-MDT0001 complete [ 585.960285] Lustre: server umount lustre-OST0000 complete [ 607.906640] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 617.128596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 617.507985] LustreError: 23689:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 617.577865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 617.585879] Lustre: Skipped 2 previous similar messages [ 621.177880] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 629.998927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 635.154507] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 638.090557] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 644.856290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 651.505587] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 661.029674] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 661.277429] Lustre: lustre-OST0001: new disk, initializing [ 661.282485] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 661.288786] Lustre: 26006:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 663.415719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 663.440059] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 663.553179] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 669.060461] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 679.555763] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 683.581451] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 697.312886] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 697.322201] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 697.332852] Lustre: Skipped 2 previous similar messages [ 697.343430] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 697.350141] Lustre: Skipped 2 previous similar messages [ 701.854820] Lustre: server umount lustre-MDT0000 complete [ 704.489758] LustreError: 23685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 704.509418] LustreError: 23685:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 705.548544] LustreError: 23671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265752 with bad export cookie 14517709543151530091 [ 705.554550] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 705.561782] LustreError: 23671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 706.158741] Lustre: server umount lustre-MDT0001 complete [ 720.548522] Lustre: server umount lustre-OST0000 complete [ 734.677201] Lustre: server umount lustre-OST0001 complete [ 752.135307] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 764.159938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 764.708131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 764.713299] Lustre: Skipped 3 previous similar messages [ 769.178257] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 770.021799] LustreError: 29029:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 770.053056] LustreError: 29029:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 777.869333] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 782.231989] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 785.633174] Lustre: 30176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 793.201249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 800.327287] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 809.831715] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 816.130404] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 823.358570] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 826.670952] Lustre: 32054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 834.538180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 834.542551] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 834.553221] Lustre: Skipped 3 previous similar messages [ 834.556182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 834.559162] Lustre: Skipped 4 previous similar messages [ 838.213617] Lustre: server umount lustre-MDT0000 complete [ 842.088384] LustreError: 29010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789265889 with bad export cookie 14517709543151531393 [ 842.099973] LustreError: 29010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 842.104809] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 842.555154] Lustre: server umount lustre-MDT0001 complete [ 857.176067] Lustre: server umount lustre-OST0000 complete [ 871.482956] Lustre: server umount lustre-OST0001 complete [ 879.591593] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 22:18:46 (1789265926) [ 909.086093] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 921.301732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 921.749831] LustreError: 34859:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 921.770396] LustreError: 34859:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 925.705687] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 932.346724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 937.671585] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 941.346705] Lustre: 36004:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 948.341700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 957.028805] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 965.123043] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 972.211974] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 979.851388] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 983.723351] Lustre: 37886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 988.129765] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 988.145244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 988.166513] Lustre: Skipped 6 previous similar messages [ 988.178015] Lustre: Skipped 6 previous similar messages [ 994.060517] Lustre: server umount lustre-MDT0000 complete [ 997.057491] LustreError: 34839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266044 with bad export cookie 14517709543151532492 [ 997.061542] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 997.070207] LustreError: 34839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 997.305709] Lustre: server umount lustre-MDT0001 complete [ 1011.135587] Lustre: server umount lustre-OST0000 complete [ 1014.240077] Lustre: 3648:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789266045/real 1789266045] req@ffff891f01f25880 x1876180492167168/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789266061 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1016.036300] Lustre: server umount lustre-OST0001 complete [ 1027.505625] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 22:21:13 (1789266073) [ 1045.573638] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1055.405660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1055.908326] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1055.918823] Lustre: Skipped 7 previous similar messages [ 1060.810570] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1069.167257] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1074.626248] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1077.569591] Lustre: 41179:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1084.366645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1091.526553] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1100.517993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1107.385386] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1115.063721] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1118.531964] Lustre: 43056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1119.464455] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1119.468170] Lustre: Skipped 1 previous similar message [ 1125.857436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1125.868908] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1125.889437] Lustre: Skipped 1 previous similar message [ 1128.938221] Lustre: server umount lustre-MDT0000 complete [ 1132.483238] LustreError: 43059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266179 with bad export cookie 14517709543151533605 [ 1132.495708] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1132.499455] LustreError: 43059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1138.991582] Lustre: server umount lustre-MDT0001 complete [ 1147.432151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1162.563697] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1166.762450] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1169.604615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1173.792840] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1175.022965] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1175.042489] Lustre: Skipped 1 previous similar message [ 1180.858334] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1182.175889] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1187.459355] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1188.919867] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1195.498353] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1199.477525] Lustre: server umount lustre-MDT0000 complete [ 1200.613992] LustreError: 45011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1200.635516] LustreError: 45011:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 25 previous similar messages [ 1203.024675] LustreError: 40011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266250 with bad export cookie 14517709543151534697 [ 1203.033325] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1203.047852] LustreError: 40011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1203.360143] Lustre: server umount lustre-MDT0001 complete [ 1218.149725] Lustre: server umount lustre-OST0000 complete [ 1232.552124] Lustre: server umount lustre-OST0001 complete [ 1240.371830] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 22:24:47 (1789266287) [ 1257.817624] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1266.970217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1271.657693] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1282.239550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1287.881106] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1291.652955] Lustre: 48815:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1297.825976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1303.609445] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1311.842588] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1318.799111] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1325.717667] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1329.388478] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1341.143837] Lustre: server umount lustre-MDT0000 complete [ 1343.970070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1344.578412] LustreError: 47645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266391 with bad export cookie 14517709543151535523 [ 1344.580467] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1344.584825] LustreError: 47645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1344.970751] Lustre: server umount lustre-MDT0001 complete [ 1358.420916] Lustre: server umount lustre-OST0000 complete [ 1370.922416] Lustre: server umount lustre-OST0001 complete [ 1381.086446] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 22:27:08 (1789266428) [ 1383.894561] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1385.462160] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 22:27:12 (1789266432) [ 1394.016608] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 1402.516893] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1424.612491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1424.914721] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1425.012303] Lustre: lustre-MDT0000: new disk, initializing [ 1425.090215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1429.765602] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1438.717821] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1445.183443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1445.610825] Lustre: lustre-OST0000: new disk, initializing [ 1445.614526] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1445.620347] Lustre: Skipped 1 previous similar message [ 1445.628299] Lustre: 54893:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1447.199762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1447.213127] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1447.276032] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1452.328359] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1462.635043] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1465.234896] Lustre: DEBUG MARKER: create test files [ 1493.475279] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1493.491294] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1493.509169] Lustre: Skipped 13 previous similar messages [ 1493.519215] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1493.527169] Lustre: Skipped 19 previous similar messages [ 1498.056260] Lustre: server umount lustre-OST0000 complete [ 1501.594591] Lustre: server umount lustre-MDT0000 complete [ 1511.584754] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 1514.047050] Key type lgssc unregistered [ 1514.275963] LNet: 56521:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1514.283789] LNetError: 56521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1514.295807] LNet: Removed LNI 192.168.204.144@tcp [ 1515.027181] Key type .llcrypt unregistered [ 1515.030726] Key type ._llcrypt unregistered [ 1537.075862] Key type ._llcrypt registered [ 1537.080273] Key type .llcrypt registered [ 1537.194357] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1538.255312] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1538.276045] alg: No test for adler32 (adler32-zlib) [ 1539.302333] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 1539.493979] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 1541.143122] Key type lgssc registered [ 1542.363759] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1550.329900] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1550.354617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1551.869297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1556.875035] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1564.735715] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1571.833540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1572.106450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1577.279216] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1577.390814] Lustre: 59227:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1577.394951] Lustre: 59227:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1577.399400] Lustre: 59227:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1577.402651] Lustre: 59227:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1577.406925] Lustre: 59227:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1577.410213] Lustre: 59227:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1578.765836] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1586.983462] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1592.093535] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1598.346598] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1633.764050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1633.771975] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1633.785319] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1636.201627] Lustre: server umount lustre-OST0000 complete [ 1639.682654] Lustre: server umount lustre-MDT0000 complete [ 1648.766777] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 1651.432864] Key type lgssc unregistered [ 1651.776788] LNet: 60865:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1651.788431] LNetError: 60865:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1651.814604] LNet: Removed LNI 192.168.204.144@tcp [ 1652.575756] Key type .llcrypt unregistered [ 1652.588131] Key type ._llcrypt unregistered [ 1680.045657] Key type ._llcrypt registered [ 1680.047888] Key type .llcrypt registered [ 1680.209753] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1681.183423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1681.233086] alg: No test for adler32 (adler32-zlib) [ 1682.333730] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 1682.698943] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 1684.409646] Key type lgssc registered [ 1685.709868] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1693.250804] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1693.271932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1694.769722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1699.142482] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1705.960629] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1711.595081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1711.889924] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1718.031517] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1720.136066] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1720.254296] Lustre: 63682:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1720.266654] Lustre: 63682:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1720.272292] Lustre: 63682:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1720.276666] Lustre: 63682:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1720.287824] Lustre: 63682:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1720.294956] Lustre: 63682:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1725.575944] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1730.818331] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1740.771405] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1740.781854] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1740.796629] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1745.041282] Lustre: server umount lustre-OST0000 complete [ 1748.382602] Lustre: server umount lustre-MDT0000 complete [ 1757.366747] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 1760.259721] Key type lgssc unregistered [ 1760.545792] LNet: 65237:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1760.555698] LNetError: 65237:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1760.570531] LNet: Removed LNI 192.168.204.144@tcp [ 1761.405663] Key type .llcrypt unregistered [ 1761.408598] Key type ._llcrypt unregistered [ 1778.439518] Key type ._llcrypt registered [ 1778.441454] Key type .llcrypt registered [ 1778.555446] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 1794.891541] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1796.285649] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1796.319468] alg: No test for adler32 (adler32-zlib) [ 1797.341681] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 1797.584978] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 1799.271268] Key type lgssc registered [ 1800.492782] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1833.359456] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1833.369463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1834.579114] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1834.604743] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1834.680949] Lustre: lustre-MDT0000: new disk, initializing [ 1834.751690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1834.767849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1838.910632] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1847.506996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1847.629954] Lustre: 68390:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1847.674793] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1847.681149] Lustre: Skipped 1 previous similar message [ 1847.734147] Lustre: lustre-MDT0001: new disk, initializing [ 1847.806083] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1847.852449] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1847.866462] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1851.516053] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1860.261702] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1866.858320] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1873.879385] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1874.118991] Lustre: lustre-OST0000: new disk, initializing [ 1874.121762] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1874.125323] Lustre: 70358:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1874.185649] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1878.862398] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1881.173204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1881.178573] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1881.264525] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1886.798607] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1891.081468] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1891.353730] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1896.874832] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1897.111483] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1902.050093] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1902.053768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1902.062868] LustreError: Skipped 1 previous similar message [ 1902.079915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1902.087307] Lustre: Skipped 1 previous similar message [ 1907.175681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1907.183364] Lustre: Skipped 1 previous similar message [ 1912.300119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1912.307552] Lustre: Skipped 1 previous similar message [ 1913.311470] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1913.557555] Lustre: server umount lustre-OST0000 complete [ 1919.456820] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1919.467062] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1919.484023] Lustre: Skipped 1 previous similar message [ 1919.503466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1922.530991] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1927.648490] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1927.653635] Lustre: Skipped 3 previous similar messages [ 1932.255221] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1932.592886] Lustre: server umount lustre-MDT0000 complete [ 1937.499423] LustreError: 68381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789266984 with bad export cookie 14561221697829327478 [ 1937.507026] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1937.514063] LustreError: 68381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1938.121918] Lustre: server umount lustre-MDT0001 complete [ 1947.972683] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:36:34 (1789266994) [ 1958.962324] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1999.894194] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 2009.535215] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2047.604692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2047.823929] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2047.845906] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2047.898343] Lustre: lustre-MDT0000: new disk, initializing [ 2047.973986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2048.003784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2052.348778] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2062.779968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2062.915990] Lustre: 76378:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2062.965622] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2062.969077] Lustre: Skipped 1 previous similar message [ 2063.024437] Lustre: lustre-MDT0001: new disk, initializing [ 2063.085845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2063.123082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2063.133696] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2067.128133] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2076.273687] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2082.233988] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2089.855339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2090.156494] Lustre: lustre-OST0000: new disk, initializing [ 2090.159920] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2090.165341] Lustre: 78349:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2090.251387] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2096.622635] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2098.305245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2098.327533] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2098.482065] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2106.700620] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2112.280425] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2112.591790] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2118.570369] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2118.941336] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2124.260955] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2124.289409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2124.325335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2124.335948] Lustre: Skipped 1 previous similar message [ 2127.503388] Lustre: server umount lustre-OST0000 complete [ 2134.496334] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2134.502954] Lustre: Skipped 1 previous similar message [ 2143.714297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2143.730819] Lustre: Skipped 5 previous similar messages [ 2145.249152] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2145.468154] Lustre: server umount lustre-MDT0000 complete [ 2148.832644] LustreError: 76388:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2148.879843] LustreError: 76388:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2150.572666] LustreError: 76372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267197 with bad export cookie 14561221697829328360 [ 2150.575016] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2150.596176] LustreError: 76372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2150.974979] Lustre: server umount lustre-MDT0001 complete [ 2162.669567] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:40:09 (1789267209) [ 2170.555259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2171.034936] LustreError: 80764:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2171.161370] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2176.024072] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2176.482161] LustreError: 80765:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2181.608814] LustreError: 80764:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2185.341777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2189.880599] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2198.250282] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2203.770161] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2211.401160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2211.993531] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2212.009255] Lustre: Skipped 1 previous similar message [ 2219.358628] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2228.941440] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2233.752372] Lustre: 83435:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting e41b80ed-c9ae-41fa-b868-8204236c2b20 at adminstrative request [ 2235.875686] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2235.905522] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2246.129831] Lustre: MGS: Client e41b80ed-c9ae-41fa-b868-8204236c2b20 (at 0@lo) reconnecting [ 2246.139853] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xca13d9fc4ad3d95a to 0xca13d9fc4ad3dbfa [ 2246.151906] Lustre: MGC192.168.204.144@tcp: Connection restored to 0@lo (at 0@lo) [ 2271.409927] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2271.617634] Lustre: lustre-OST0001: new disk, initializing [ 2271.623225] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2271.633612] Lustre: 83832:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2278.012625] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2279.512749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2279.528936] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2279.652530] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2287.597772] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2295.268374] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2295.270467] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2295.285356] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2295.293990] Lustre: Skipped 2 previous similar messages [ 2295.304574] LustreError: Skipped 2 previous similar messages [ 2296.862780] Lustre: server umount lustre-OST0001 complete [ 2300.389372] LustreError: 83469:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2300.405317] LustreError: 83469:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2302.433128] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2306.586506] Lustre: server umount lustre-OST0000 complete [ 2316.090413] Lustre: server umount lustre-MDT0000 complete [ 2320.043405] LustreError: 80744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267367 with bad export cookie 14561221697829329914 [ 2320.054389] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2320.059178] LustreError: 80744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2320.411050] Lustre: server umount lustre-MDT0001 complete [ 2329.835587] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:42:56 (1789267376) [ 2331.548349] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2333.983297] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:43:00 (1789267380) [ 2335.602748] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2337.883819] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:43:04 (1789267384) [ 2339.898969] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2342.184648] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:43:08 (1789267388) [ 2344.098644] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2346.411488] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:43:13 (1789267393) [ 2347.796915] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2353.547887] loop: module loaded [ 2353.583259] loop0: detected capacity change from 0 to 4294967296 [ 2353.628161] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2355.324661] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2358.955786] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2364.004667] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2395.127223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2395.205618] LustreError: 87955:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2395.211896] LustreError: 87955:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2395.221208] LustreError: 87955:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2395.234496] LustreError: 87955:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2416.202913] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 2427.015340] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2467.035256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2467.329459] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2467.371541] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2467.513100] Lustre: lustre-MDT0000: new disk, initializing [ 2467.674807] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2467.687217] Lustre: Skipped 1 previous similar message [ 2467.705778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2472.672825] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2483.431924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2483.571943] Lustre: 90821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2483.760969] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2488.777223] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2500.811834] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2507.787268] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2515.505891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2515.914723] Lustre: 92790:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2522.835280] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2525.751849] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2525.915476] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2533.838049] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2540.394341] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2540.706394] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2546.816218] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2547.164907] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2551.778507] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2551.792061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2551.797719] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2551.797727] Lustre: Skipped 9 previous similar messages [ 2551.799329] Lustre: Skipped 5 previous similar messages [ 2551.842484] LustreError: Skipped 1 previous similar message [ 2555.835724] Lustre: server umount lustre-OST0000 complete [ 2574.303190] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2574.606677] Lustre: server umount lustre-MDT0000 complete [ 2577.393912] LustreError: 90833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2577.408362] LustreError: 90833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2579.292893] LustreError: 90815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267626 with bad export cookie 14561221697829330530 [ 2579.302453] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2579.309118] LustreError: 90815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2579.869474] Lustre: server umount lustre-MDT0001 complete [ 2590.154304] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:47:16 (1789267636) [ 2598.064762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2598.623885] LustreError: 95207:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2598.724244] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2598.734638] Lustre: Skipped 2 previous similar messages [ 2603.888303] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2612.988867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2618.205641] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2626.340440] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2633.143811] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2641.109189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2649.722104] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2659.203210] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2665.231651] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2665.521863] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2670.485235] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2670.776293] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2677.881283] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2678.082352] Lustre: lustre-OST0001: new disk, initializing [ 2678.086895] Lustre: Skipped 2 previous similar messages [ 2678.092543] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2678.098983] Lustre: Skipped 3 previous similar messages [ 2678.107313] Lustre: 98582:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2683.522626] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2683.538312] Lustre: Skipped 2 previous similar messages [ 2683.550785] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2683.702447] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2686.003975] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2696.027661] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2700.969701] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2701.176624] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2706.874724] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2707.099528] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2728.823805] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2758.248846] Lustre: Failing over lustre-MDT0000 [ 2758.878789] Lustre: server umount lustre-MDT0000 complete [ 2760.429796] LustreError: 96571:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2760.453231] LustreError: 96571:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2760.677684] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2760.705758] Lustre: Skipped 4 previous similar messages [ 2777.055253] Lustre: 66613:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789267808/real 1789267808] req@0000000006b3e87c x1876182270496768/t0(0) o400->MGC192.168.204.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789267824 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2777.084450] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2784.416198] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2784.424301] LDISKFS-fs (dm-0): recovery complete [ 2784.434485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2786.656702] LustreError: 101404:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2786.661671] LustreError: 101404:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000932aa6c0 x1876182270502528/t0(0) o250->MGC192.168.204.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1789267834 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2786.672861] LustreError: 101404:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2787.312207] LustreError: 66610:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000932aa6c0 x1876182270506240/t0(0) o250->MGC192.168.204.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2787.744827] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2791.129402] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2791.134110] Lustre: 95203:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2792.935589] Lustre: 95207:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2792.953791] Lustre: 95207:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2792.972328] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2793.076307] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2793.423185] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2793.767566] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2793.788127] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2793.833046] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2794.114337] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2794.124765] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2794.150979] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2794.160640] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2794.810159] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2794.824390] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2795.143838] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2795.167130] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2796.166489] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2796.200361] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2796.869028] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2796.902697] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2797.224483] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2797.253712] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2800.166975] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2800.182783] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 11 previous similar messages [ 2801.158942] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2801.171311] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2801.495161] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2801.511831] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2808.236067] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2808.275982] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2809.248443] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2809.261502] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2809.593205] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2809.597114] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2824.265378] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2824.284332] Lustre: 101434:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 2825.564770] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2825.576184] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2825.903155] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2825.915384] LustreError: 101434:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2851.191505] Lustre: 101434:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2851.202318] Lustre: 101434:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2851.222188] Lustre: 101434:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 3c5ef461-010e-4941-9332-99b7cbbac640@192.168.204.44@tcp [ 2851.237347] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2851.242788] Lustre: 101434:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000f6f12dc3 x1876182254557056/t0(8589934777) o101->3c5ef461-010e-4941-9332-99b7cbbac640@192.168.204.44@tcp:204/0 lens 592/0 e 0 to 0 dl 1789267909 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2851.298992] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2851.306600] Lustre: lustre-MDT0000: Denying connection for new client 3c5ef461-010e-4941-9332-99b7cbbac640 (at 192.168.204.44@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2851.343726] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2851.397620] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2851.411628] Lustre: Skipped 2 previous similar messages [ 2851.411989] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2851.476115] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2851.481113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2867.683234] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2867.690688] LustreError: Skipped 1 previous similar message [ 2867.692454] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2867.710836] Lustre: Skipped 1 previous similar message [ 2867.720204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2867.734402] Lustre: Skipped 8 previous similar messages [ 2873.704069] Lustre: server umount lustre-OST0000 complete [ 2874.852541] LustreError: 98822:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2874.882518] LustreError: 98822:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 40 previous similar messages [ 2883.644673] Lustre: server umount lustre-OST0001 complete [ 2894.320195] Lustre: server umount lustre-MDT0000 complete [ 2898.329859] LustreError: 95189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789267945 with bad export cookie 14561221697829381749 [ 2898.340618] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2898.357970] LustreError: 95189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2898.815947] Lustre: server umount lustre-MDT0001 complete [ 2907.789381] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:52:35 (1789267955) [ 2914.248902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2914.252164] Lustre: *** cfs_fail_loc=197, val=0*** [ 2929.888352] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2935.009434] LustreError: 103407:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.144@tcp: failed processing log, type 4: rc = -110 [ 2940.150479] Lustre: *** cfs_fail_loc=197, val=0*** [ 2940.152948] Lustre: Skipped 65 previous similar messages [ 2960.738366] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2960.755085] Lustre: Skipped 4 previous similar messages [ 2968.105310] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2976.724208] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2979.428591] Lustre: server umount lustre-OST0000 complete [ 2989.330840] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:53:56 (1789268036) [ 3011.312428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3011.780649] LustreError: 105028:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3011.796410] LustreError: 105028:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3016.811829] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3026.360991] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3030.779928] LustreError: 105059:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3030.801741] LustreError: 105059:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3030.810482] LustreError: 105059:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3031.232927] Lustre: server umount lustre-MDT0000 complete [ 3067.538435] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 3080.421626] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3122.911270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3123.154176] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3123.191605] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3123.297133] Lustre: lustre-MDT0000: new disk, initializing [ 3123.406517] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3128.557207] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3140.414073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3140.498862] Lustre: 109234:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3140.815845] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3146.688931] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3158.479988] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3166.156382] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3173.909891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3174.312312] Lustre: 111202:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3181.301662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3181.690740] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3193.003426] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3199.143328] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3199.442343] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3205.351973] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3205.655585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3212.267819] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3212.268268] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3212.281632] LustreError: Skipped 4 previous similar messages [ 3212.301856] Lustre: Skipped 6 previous similar messages [ 3212.309558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3212.320655] Lustre: Skipped 10 previous similar messages [ 3215.085317] Lustre: server umount lustre-OST0000 complete [ 3234.271444] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3234.489764] Lustre: server umount lustre-MDT0000 complete [ 3237.830914] LustreError: 109227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789268285 with bad export cookie 14561221697829392403 [ 3237.841022] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3238.113392] Lustre: server umount lustre-MDT0001 complete [ 3247.111855] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:58:14 (1789268294) [ 3259.098754] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3263.231140] Key type lgssc unregistered [ 3263.682357] LNet: 113691:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3263.695287] LNetError: 113691:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3263.722303] LNet: Removed LNI 192.168.204.144@tcp [ 3264.621597] Key type .llcrypt unregistered [ 3264.623899] Key type ._llcrypt unregistered [ 3283.566639] Key type ._llcrypt registered [ 3283.568158] Key type .llcrypt registered [ 3283.648168] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 3300.960841] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3302.247333] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3302.423740] alg: No test for adler32 (adler32-zlib) [ 3303.619285] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 3303.927439] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3305.663241] Key type lgssc registered [ 3306.999257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3352.910072] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3352.945984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3354.311448] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3354.372697] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3354.561992] Lustre: lustre-MDT0000: new disk, initializing [ 3354.746029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3354.775375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3360.071412] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3371.347507] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3371.638638] Lustre: lustre-OST0000: new disk, initializing [ 3371.642770] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3371.650796] Lustre: Skipped 1 previous similar message [ 3371.659607] Lustre: 117861:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3371.755897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3377.804470] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3377.822155] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3377.869524] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3379.164889] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3391.613543] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3391.964525] Lustre: lustre-OST0001: new disk, initializing [ 3391.969793] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3392.002603] Lustre: 118725:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3392.215576] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3399.411432] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3399.421677] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3399.455935] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3401.134076] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3415.024265] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3415.027306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3415.052128] Lustre: Skipped 1 previous similar message [ 3417.307229] Lustre: server umount lustre-MDT0000 complete [ 3425.608414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3430.790368] LustreError: 117050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789268478 with bad export cookie 9847578133112253332 [ 3430.791310] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3430.805026] LustreError: 117050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3430.909177] Lustre: server umount lustre-OST0000 complete [ 3435.048412] Lustre: server umount lustre-OST0001 complete [ 3443.442138] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 23:01:30 (1789268490) [ 3464.961355] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 23:01:52 (1789268512) [ 3474.074966] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3499.022909] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 23:02:25 (1789268545) [ 3537.028429] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 3546.851446] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3578.012938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3578.185833] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3578.222221] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3578.280288] Lustre: lustre-MDT0000: new disk, initializing [ 3578.340162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3578.368816] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3582.453661] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3595.313739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3595.424647] Lustre: 126491:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3595.497656] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3595.515564] Lustre: Skipped 1 previous similar message [ 3595.594322] Lustre: lustre-MDT0001: new disk, initializing [ 3595.700302] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3595.744075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3595.762328] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3600.746876] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3610.683855] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3616.972639] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3624.426964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3624.803486] Lustre: lustre-OST0000: new disk, initializing [ 3624.813532] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3624.836585] Lustre: 128459:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3624.931307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3632.527310] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3632.711093] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3632.717254] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3632.908699] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3640.374848] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3643.730709] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3643.888583] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3649.078188] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3649.277318] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3653.600365] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3653.604534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3653.606632] LustreError: Skipped 1 previous similar message [ 3653.620393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3653.625204] Lustre: Skipped 1 previous similar message [ 3653.633426] Lustre: Skipped 1 previous similar message [ 3657.797947] Lustre: server umount lustre-OST0000 complete [ 3662.304705] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3662.320957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3662.341674] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3662.350219] Lustre: Skipped 1 previous similar message [ 3668.644224] Lustre: server umount lustre-MDT0000 complete [ 3673.116751] LustreError: 126481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789268720 with bad export cookie 9847578133112254557 [ 3673.128327] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3673.130255] LustreError: 126481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3673.839386] Lustre: server umount lustre-MDT0001 complete [ 3684.598798] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 23:05:31 (1789268731) [ 3690.500839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3690.990506] LustreError: 130873:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3691.007890] LustreError: 130873:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3691.082133] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3696.022563] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3696.114375] LustreError: 130873:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3696.142119] LustreError: 130873:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3701.221642] LustreError: 130874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3706.337498] LustreError: 130873:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3706.639756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3713.411932] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3721.892625] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3727.755846] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3735.132624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3735.602093] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3735.607954] Lustre: Skipped 1 previous similar message [ 3743.534301] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3751.188981] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3759.179914] Lustre: *** cfs_fail_loc=159, val=0*** [ 3774.746295] Lustre: lustre-MDT0000: Client 08b5bcef-04d8-430e-8f3e-b61c3578e684 (at 192.168.204.44@tcp) reconnecting [ 3776.596815] Lustre: *** cfs_fail_loc=159, val=0*** [ 3776.607066] Lustre: Skipped 5 previous similar messages [ 3793.198994] Lustre: lustre-MDT0000: Client 08b5bcef-04d8-430e-8f3e-b61c3578e684 (at 192.168.204.44@tcp) reconnecting [ 3803.113790] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3803.132743] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3803.149074] Lustre: Skipped 1 previous similar message [ 3803.155516] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3803.162768] Lustre: Skipped 2 previous similar messages [ 3805.828403] Lustre: server umount lustre-OST0000 complete [ 3814.370677] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3814.377808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3814.409165] Lustre: Skipped 2 previous similar messages [ 3814.419735] Lustre: Skipped 3 previous similar messages [ 3816.647880] Lustre: server umount lustre-MDT0000 complete [ 3819.497699] LustreError: 130870:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3819.521933] LustreError: 130870:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3820.735736] LustreError: 130855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789268868 with bad export cookie 9847578133112255432 [ 3820.739269] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3820.750796] LustreError: 130855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3821.304926] Lustre: server umount lustre-MDT0001 complete [ 3833.958296] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3837.320472] Key type lgssc unregistered [ 3837.814815] LNet: 134653:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3837.822476] LNetError: 134653:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3837.858107] LNet: Removed LNI 192.168.204.144@tcp [ 3838.933342] Key type .llcrypt unregistered [ 3838.936790] Key type ._llcrypt unregistered [ 3867.063338] Key type ._llcrypt registered [ 3867.064873] Key type .llcrypt registered [ 3867.202324] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3868.437761] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3868.681595] alg: No test for adler32 (adler32-zlib) [ 3869.996172] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 3870.373729] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3872.071178] Key type lgssc registered [ 3873.450984] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3881.374115] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3881.394709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3882.938947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3888.619520] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3893.218637] LustreError: 136066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3893.235293] LustreError: 136066:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3898.337749] LustreError: 136067:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3898.442869] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3898.991508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3904.501387] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3913.057232] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3919.908398] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3927.425061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3927.984152] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3929.032930] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3936.654994] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3944.947417] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3956.081933] Lustre: *** cfs_fail_loc=159, val=0*** [ 3972.411113] Lustre: lustre-MDT0000: Client ba197229-6e82-4b02-ba66-99abf010cde0 (at 192.168.204.44@tcp) reconnecting [ 3975.254488] Lustre: *** cfs_fail_loc=159, val=0*** [ 3990.787524] Lustre: lustre-MDT0001: Client ba197229-6e82-4b02-ba66-99abf010cde0 (at 192.168.204.44@tcp) reconnecting [ 3992.403882] Lustre: *** cfs_fail_loc=159, val=0*** [ 3992.413468] Lustre: Skipped 4 previous similar messages [ 4007.693074] Lustre: lustre-MDT0001: Client ba197229-6e82-4b02-ba66-99abf010cde0 (at 192.168.204.44@tcp) reconnecting [ 4011.068680] Lustre: 139104:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4013.747751] Lustre: *** cfs_fail_loc=159, val=0*** [ 4013.751041] Lustre: Skipped 3 previous similar messages [ 4030.262885] Lustre: lustre-MDT0000: Client 84fff136-f666-47c5-abd9-6e81fb2bd0c0 (at 192.168.204.44@tcp) reconnecting [ 4032.061286] Lustre: *** cfs_fail_loc=159, val=0*** [ 4032.072757] Lustre: Skipped 9 previous similar messages [ 4047.636512] Lustre: lustre-MDT0000: Client 84fff136-f666-47c5-abd9-6e81fb2bd0c0 (at 192.168.204.44@tcp) reconnecting [ 4049.382557] Lustre: 139396:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4056.039171] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4056.045788] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4056.065454] LustreError: Skipped 1 previous similar message [ 4056.114384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4057.062897] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4057.071906] LustreError: Skipped 4 previous similar messages [ 4057.080080] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4057.104368] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4061.749793] Lustre: server umount lustre-OST0000 complete [ 4067.808573] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4067.826056] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4067.863166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4067.874084] Lustre: Skipped 2 previous similar messages [ 4072.963826] Lustre: server umount lustre-MDT0000 complete [ 4073.447342] LustreError: 139346:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4073.487391] LustreError: 139346:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4077.560584] LustreError: 139409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789269124 with bad export cookie 2883413496829135344 [ 4077.563136] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4077.573703] LustreError: 139409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4078.104801] Lustre: server umount lustre-MDT0001 complete [ 4092.337641] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 4095.946833] Key type lgssc unregistered [ 4096.304702] LNet: 140450:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4096.311270] LNetError: 140450:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4096.327797] LNet: Removed LNI 192.168.204.144@tcp [ 4097.253466] Key type .llcrypt unregistered [ 4097.260583] Key type ._llcrypt unregistered [ 4126.942095] Key type ._llcrypt registered [ 4126.945283] Key type .llcrypt registered [ 4127.097545] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 4128.839053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4128.896126] alg: No test for adler32 (adler32-zlib) [ 4130.504744] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 4130.958679] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 4132.799235] Key type lgssc registered [ 4134.708979] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4143.299247] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4143.329235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4144.795419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4150.269592] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4155.363891] LustreError: 141862:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4155.387209] LustreError: 141862:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4159.812551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4160.323684] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4165.915748] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4174.381492] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4180.767174] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4188.750466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4189.119590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4190.157837] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4196.891535] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4205.527581] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4205.929618] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4213.124211] Lustre: 144447:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4216.251670] Lustre: 144496:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4218.183521] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4219.250828] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4223.701633] Lustre: 144642:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4224.734222] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4225.736336] Lustre: 144741:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4231.136664] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4231.138078] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4231.153492] LustreError: Skipped 1 previous similar message [ 4231.157732] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4231.169769] Lustre: Skipped 1 previous similar message [ 4236.258996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4236.270054] Lustre: Skipped 2 previous similar messages [ 4237.088169] Lustre: server umount lustre-OST0000 complete [ 4242.411977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4242.422349] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4242.441938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4246.503262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4246.515943] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.213802] Lustre: server umount lustre-MDT0000 complete [ 4251.617972] LustreError: 141861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4251.650130] LustreError: 141861:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4252.731396] LustreError: 141842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789269300 with bad export cookie 8683620425655930689 [ 4252.741475] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4252.751143] LustreError: 141842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4253.163201] Lustre: server umount lustre-MDT0001 complete [ 4265.269951] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 4268.596832] Key type lgssc unregistered [ 4269.035476] LNet: 145801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4269.046465] LNetError: 145801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4270.120914] LNet: Removed LNI 192.168.204.144@tcp [ 4271.242896] Key type .llcrypt unregistered [ 4271.244149] Key type ._llcrypt unregistered [ 4298.536392] Key type ._llcrypt registered [ 4298.545738] Key type .llcrypt registered [ 4298.760707] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 4300.263443] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4300.419599] alg: No test for adler32 (adler32-zlib) [ 4301.587281] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 4301.869486] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 4303.583200] Key type lgssc registered [ 4305.083837] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4312.497721] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4312.525672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4313.916469] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4319.294153] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4324.323869] LustreError: 147210:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4324.338942] LustreError: 147210:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4328.999618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4329.447262] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4329.676748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4335.080066] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4344.351236] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4351.013414] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4358.736879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4359.065276] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4365.908624] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4373.828455] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4374.468825] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4383.697000] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4389.536528] Lustre: *** cfs_fail_loc=159, val=0*** [ 4406.048691] Lustre: lustre-MDT0000: Client 47c31e23-b06b-4f99-aa6d-5c0bb7177967 (at 192.168.204.44@tcp) reconnecting [ 4414.434817] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4414.435754] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4414.441075] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4414.462499] Lustre: Skipped 1 previous similar message [ 4418.837832] Lustre: server umount lustre-OST0000 complete [ 4424.676553] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4424.685733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4424.698029] Lustre: Skipped 1 previous similar message [ 4424.714517] Lustre: Skipped 2 previous similar messages [ 4429.792932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4429.800291] Lustre: Skipped 1 previous similar message [ 4430.429281] Lustre: server umount lustre-MDT0000 complete [ 4433.896656] LustreError: 147205:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4433.926366] LustreError: 147205:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4435.047851] LustreError: 147951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789269482 with bad export cookie 11916649689731926554 [ 4435.058776] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4435.069636] LustreError: 147951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4435.569772] Lustre: server umount lustre-MDT0001 complete [ 4447.738470] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 4450.904295] Key type lgssc unregistered [ 4451.222817] LNet: 150896:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4451.234554] LNetError: 150896:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4451.269774] LNet: Removed LNI 192.168.204.144@tcp [ 4452.203709] Key type .llcrypt unregistered [ 4452.206524] Key type ._llcrypt unregistered [ 4478.196860] Key type ._llcrypt registered [ 4478.199957] Key type .llcrypt registered [ 4478.348278] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 4479.752986] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4479.921277] alg: No test for adler32 (adler32-zlib) [ 4480.928715] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 4481.152106] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 4482.855495] Key type lgssc registered [ 4484.838824] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4493.558433] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4493.586570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4495.141847] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4500.724247] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4505.576423] LustreError: 152308:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4505.598651] LustreError: 152308:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4510.484860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4510.691123] LustreError: 152309:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4510.982435] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4516.542524] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4525.807560] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4531.751435] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4538.571387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4539.162629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4540.268975] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4546.541677] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4554.615369] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4556.607744] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4575.202832] Lustre: lustre-OST0000: evicting ee4e8834-c2fc-43c2-9bfc-f01520fd41f1 (at 192.168.204.44@tcp) by administrative request [ 4575.210573] Lustre: Skipped 1 previous similar message [ 4606.944896] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4606.959135] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4606.979332] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4607.970939] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4607.972891] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4607.985909] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4608.006379] Lustre: Skipped 1 previous similar message [ 4613.090105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4613.094020] Lustre: Skipped 1 previous similar message [ 4618.207306] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4618.234822] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4618.245729] Lustre: Skipped 1 previous similar message [ 4618.455559] Lustre: server umount lustre-OST0000 complete [ 4623.330673] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4623.335622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4623.342062] Lustre: Skipped 1 previous similar message [ 4629.256703] Lustre: server umount lustre-MDT0000 complete [ 4633.274603] LustreError: 152290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789269680 with bad export cookie 13542196814212121906 [ 4633.278934] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4633.286932] LustreError: 152290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4633.695968] Lustre: server umount lustre-MDT0001 complete [ 4646.593094] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 4649.645415] Key type lgssc unregistered [ 4650.103561] LNet: 156388:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4650.119907] LNetError: 156388:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4650.142928] LNet: Removed LNI 192.168.204.144@tcp [ 4651.092210] Key type .llcrypt unregistered [ 4651.096347] Key type ._llcrypt unregistered [ 4739.073678] Key type ._llcrypt registered [ 4739.076859] Key type .llcrypt registered [ 4739.201632] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 4740.488259] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4740.589634] alg: No test for adler32 (adler32-zlib) [ 4741.696940] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 4741.945752] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 4743.623136] Key type lgssc registered [ 4745.179933] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4754.874651] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4754.902093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4756.543483] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4761.421369] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4765.665393] LustreError: 158875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4765.681782] LustreError: 158875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4770.788739] LustreError: 158876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4771.397135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4772.006803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4777.727989] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4786.259927] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4793.129608] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4800.831724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4800.934659] LustreError: 160704:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4801.333252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4806.656632] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4807.944625] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4816.834944] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4816.895247] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4827.104524] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4827.125249] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4827.141188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4827.148014] Lustre: Skipped 1 previous similar message [ 4827.162566] Lustre: Skipped 1 previous similar message [ 4832.228380] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4832.234576] Lustre: Skipped 1 previous similar message [ 4833.040984] Lustre: server umount lustre-OST0000 complete [ 4837.347796] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4837.368110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4837.373978] Lustre: Skipped 1 previous similar message [ 4842.467992] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4842.479072] Lustre: Skipped 1 previous similar message [ 4842.611135] Lustre: server umount lustre-MDT0000 complete [ 4847.584596] LustreError: 158875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4847.594270] LustreError: 158875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4847.757977] LustreError: 158857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789269895 with bad export cookie 14787695511763013680 [ 4847.770241] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4847.778514] LustreError: 158857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4848.298802] Lustre: server umount lustre-MDT0001 complete [ 4862.634722] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 4865.788665] Key type lgssc unregistered [ 4866.119979] LNet: 162418:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4866.129561] LNetError: 162418:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4866.160744] LNet: Removed LNI 192.168.204.144@tcp [ 4867.162565] Key type .llcrypt unregistered [ 4867.166502] Key type ._llcrypt unregistered [ 4894.376221] Key type ._llcrypt registered [ 4894.378835] Key type .llcrypt registered [ 4894.446121] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 4895.243046] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4895.442436] alg: No test for adler32 (adler32-zlib) [ 4896.678342] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 4896.994723] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 4898.679170] Key type lgssc registered [ 4900.164773] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4916.236918] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 23:26:02 (1789269962) [ 4931.692262] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 4943.530764] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 4984.163357] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4984.208314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4985.467330] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4985.497556] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4985.624291] Lustre: lustre-MDT0000: new disk, initializing [ 4985.720297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4985.756235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4989.651665] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4998.740897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4998.920575] Lustre: 166678:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4999.060710] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4999.072355] Lustre: Skipped 1 previous similar message [ 4999.156042] Lustre: lustre-MDT0001: new disk, initializing [ 4999.241817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4999.283869] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4999.294613] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5004.514268] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5015.962266] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5023.122990] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5029.004393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5029.270222] Lustre: lustre-OST0000: new disk, initializing [ 5029.272939] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5029.278730] Lustre: 168650:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5029.346324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5035.357663] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5037.719623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5037.740844] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5037.897609] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5043.636499] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5053.411496] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5053.412485] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5053.437556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5058.542030] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5058.549728] Lustre: Skipped 2 previous similar messages [ 5059.368714] Lustre: server umount lustre-OST0000 complete [ 5063.659375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5063.665477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5063.674877] Lustre: Skipped 2 previous similar messages [ 5068.769867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5068.776752] Lustre: Skipped 2 previous similar messages [ 5071.309098] Lustre: server umount lustre-MDT0000 complete [ 5075.455475] LustreError: 166672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789270122 with bad export cookie 10289851644018121976 [ 5075.469103] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5075.470207] LustreError: 166672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5075.767264] Lustre: server umount lustre-MDT0001 complete [ 5086.543775] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 5088.776585] Key type lgssc unregistered [ 5089.025286] LNet: 170557:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5089.035445] LNetError: 170557:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5089.049320] LNet: Removed LNI 192.168.204.144@tcp [ 5089.704644] Key type .llcrypt unregistered [ 5089.707453] Key type ._llcrypt unregistered [ 5111.073833] Key type ._llcrypt registered [ 5111.076387] Key type .llcrypt registered [ 5111.201609] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 5112.030837] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5112.050880] alg: No test for adler32 (adler32-zlib) [ 5113.089627] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 5113.324106] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 5114.967203] Key type lgssc registered [ 5116.264196] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5123.297700] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5123.315893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5124.730733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5129.624588] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5135.336797] LustreError: 171970:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5135.360408] LustreError: 171970:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5138.023683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5138.336373] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5142.903534] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5149.813463] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5155.695842] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5163.158420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5163.447561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5169.539981] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5170.633972] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5177.225764] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5187.492683] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5187.502173] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5187.518802] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5187.525707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.542023] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5188.425241] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5188.463496] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5188.490557] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5188.497485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.514244] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5189.432689] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5189.457710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.333945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5190.356477] Lustre: Skipped 1 previous similar message [ 5190.367257] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5190.376780] Lustre: Skipped 1 previous similar message [ 5190.381853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.407363] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5190.421206] Lustre: Skipped 1 previous similar message [ 5191.344019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.221968] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5192.232133] Lustre: Skipped 2 previous similar messages [ 5192.247308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.221692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5193.235024] Lustre: Skipped 2 previous similar messages [ 5193.241113] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5193.247428] Lustre: Skipped 2 previous similar messages [ 5193.250679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.257839] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5193.262177] Lustre: Skipped 2 previous similar messages [ 5194.185140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.196473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.061837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.044797] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5197.053470] Lustre: Skipped 4 previous similar messages [ 5197.070699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.234178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5198.245966] Lustre: Skipped 4 previous similar messages [ 5198.252043] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5198.258735] Lustre: Skipped 4 previous similar messages [ 5198.264824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.274330] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5198.278917] Lustre: Skipped 4 previous similar messages [ 5199.280408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.078962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.042851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.871452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.906627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.767079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.739248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.662862] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5205.669902] Lustre: Skipped 8 previous similar messages [ 5205.695099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.597455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5206.607443] Lustre: Skipped 8 previous similar messages [ 5206.611363] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5206.615915] Lustre: Skipped 8 previous similar messages [ 5206.621464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.629077] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5206.640118] Lustre: Skipped 8 previous similar messages [ 5207.581889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.593242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.438619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.426351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.307360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.256597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.171903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.094105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.030919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.852696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.697333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.666821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.530601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.526669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.403562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.437124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.254595] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5222.262588] Lustre: Skipped 17 previous similar messages [ 5222.278775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.254313] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5223.263355] Lustre: Skipped 17 previous similar messages [ 5223.268131] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5223.272719] Lustre: Skipped 17 previous similar messages [ 5223.279945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.288977] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5223.294345] Lustre: Skipped 17 previous similar messages [ 5224.229521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.255986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.087886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.111631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.931739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.872923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.764903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.595090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.420303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.383347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.256741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.114792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.217349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.104536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.016111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.005917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.955906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.093491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.046942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.185205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.237554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.289560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.331354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.364313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.462021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.415463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.387285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.428557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.451206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.408583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.284779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.132609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.091733] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5255.102397] Lustre: Skipped 33 previous similar messages [ 5255.119592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.935150] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5255.952120] Lustre: Skipped 33 previous similar messages [ 5255.957204] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5255.963241] Lustre: Skipped 33 previous similar messages [ 5255.967306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.980647] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5255.988943] Lustre: Skipped 33 previous similar messages [ 5256.807440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.583948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.448392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.467980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.459678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.345260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.255360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.292077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.231860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.082700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.983848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.714563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.413943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.151950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.928828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.659813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.511537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.283737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.052971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.845171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.586474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.449383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.318658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.245407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.009907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.773187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.575185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.440962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.147894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.935214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.960614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.859157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.678811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.517478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.319490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.170627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.046715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.999792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.041305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.870969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.782892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.547671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.404755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.265839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.172714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.198654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.127413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.082885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.144546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.096937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.065994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.196313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.058475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.081351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.110268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.077117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.156876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.158921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.190900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.071868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.947974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.873704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.029815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.175231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.263138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.319979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.264430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.552445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.560675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.747094] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5319.752135] Lustre: Skipped 70 previous similar messages [ 5319.771419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.679603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5320.692128] Lustre: Skipped 70 previous similar messages [ 5320.697392] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5320.701615] Lustre: Skipped 70 previous similar messages [ 5320.705943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.715400] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5320.722659] Lustre: Skipped 70 previous similar messages [ 5321.656384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.629389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.578012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.406750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.441881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.390546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.344978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.274290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.179317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.988187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.891397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.836761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.850441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.990203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.010138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.047983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.227923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.322990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.382249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.592836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.732314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.896693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.148311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.079899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.204081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.276673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.258429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.336510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.352987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.292530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.137616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.098652] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5353.103299] LustreError: 171989:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5353.115369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.016340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.845847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.780744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.766941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.646444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.724666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.644655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.764737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.674518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.744285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.648288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.847219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.966944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.878511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.246062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.338639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.408837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.633629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.690047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.623060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.612087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.665362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.617970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.619036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.615635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.778923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.590353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.462728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.318835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.130618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.044874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.019752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.875831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.859922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.910625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.681426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.540933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.427409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.353231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.210163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.994961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.965352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.893086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.882826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.893686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.887552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.766665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.803320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.856516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.794369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.685373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.508748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.488046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.406728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.323827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.451167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.523557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.490968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.532357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.597732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.482753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.534804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.521387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.677870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.639024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.456771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.137818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.051057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.933764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.717430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.734319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.680684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.580591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.518221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.466589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.368989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.242279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.393043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.455691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.717205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.030207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.398489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.666896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.902394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.037578] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5437.061838] LustreError: 171989:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5437.073572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.982577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.132410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.288786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.598635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.962830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.379366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.761477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.844695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.691795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.685575] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5448.692889] Lustre: Skipped 127 previous similar messages [ 5448.712927] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5448.724110] Lustre: Skipped 126 previous similar messages [ 5448.728321] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5448.734433] Lustre: Skipped 126 previous similar messages [ 5448.739153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.748555] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5448.757478] Lustre: Skipped 126 previous similar messages [ 5449.885660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.060453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.893731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.852149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.116416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.246816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.970604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.949179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.902238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.088848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.140213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.077305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.154745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.299226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.959217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.999835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.095942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.138164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.074980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.851968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.887360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.744148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.550753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.523035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.356796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.073053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.837800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.746184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.545983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.470732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.456930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.561719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.563587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.894118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.097142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.153465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.319725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.528345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.680568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.094095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.510604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.467214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.956971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.415382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.403768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.656715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.697119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.756583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.805574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.563284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.562158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.468727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.626907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.516502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.517614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.677222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.482760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.138649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.821429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.920362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.914080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.948329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.902158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.812064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.594672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.462873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.198214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.102204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.257465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.399263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.554905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.592301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.730715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.730636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.666321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.847373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.092461] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5527.100791] LustreError: 171989:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5527.117387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.065415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.033933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.904975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.880802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.011531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.954560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.884339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.715897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.738729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.726095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.728597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.627300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.817702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.797276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.820197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.671391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.552716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.348679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.344996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.450592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.506732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.424380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.523488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.433085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.505652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.520711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.544736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.503397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.658624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.788543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.795798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.816317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.938557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.957312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.100353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.165486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.164848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.249338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.176723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.392401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.424522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.419980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.381623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.411151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.437930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.443725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.364202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.228038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.258900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.305232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.246648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.221763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.317885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.272605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.444150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.422102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.598673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.678736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.699668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.761142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.878707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.736270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.915352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.740335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.579746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.478922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.563327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.335424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.235472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.292835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.252295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.094236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.014479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.936801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.846065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.918893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.870639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.769878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.719733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.745302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.611113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.584195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.656191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.774749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.813335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.922468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.148190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.017996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.345217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.532154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.472893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.472029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.672302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.597497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.583204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.663422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.703078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.818455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.564555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.650213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.697078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.712143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.902603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.921153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.067988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.012523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.920673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.825910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.619485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.556504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.420965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.169928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.048675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.904018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.721950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.539177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.413843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.376218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.426419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.402480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.523703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.466239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.420513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.388945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.258789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.117052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.915693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.700491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.471921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.402208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.285463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.287867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.283524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.292121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.227439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.275941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.490159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.252803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.113564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.950753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.021977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.223866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.156557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.993463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.872965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.854861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.708903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.655534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.717280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.522524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.424543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.576882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.535529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.413963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.150461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.199985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.311358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.156426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.077415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.018542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.137414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.127675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.224959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.023160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.964719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.669204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.399925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.212230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.276355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.277271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.389924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.464031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.445123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.508331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.360604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.400656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.294121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.448440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.540551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.609696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.692848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.789296] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5705.797245] Lustre: Skipped 258 previous similar messages [ 5705.832913] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5705.851521] Lustre: Skipped 258 previous similar messages [ 5705.858996] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5705.867797] Lustre: Skipped 258 previous similar messages [ 5705.871148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.888632] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5705.900590] Lustre: Skipped 258 previous similar messages [ 5707.012752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.266028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.319402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.232448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.213075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.134333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.081985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.190082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.309675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.277154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.182862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.276341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.186680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.108638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.995629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.864896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.807646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.846833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.734334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.616366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.484632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.445577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.244068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.324862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.141677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.929574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.786864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.675254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.487591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.369359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.203864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.136641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.098738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.004146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.948675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.951157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.911837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.904459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.789166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.684724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.654632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.648925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.573927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.425465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.280205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.103718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.924559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.860408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.599395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.335313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.198287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.292218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.188143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.249291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.193918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.137926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.035522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.160575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.143040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.231249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.245861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.314855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.199315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.197453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.153716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.214632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.134305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.086648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.998802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.987680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.928572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.874541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.731778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.665939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.529467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.363231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.188301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.963214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.791912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.584230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.449193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.235729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.937578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.626836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.491939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.276670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.135273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.983887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.835710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.529492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.646743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.612749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.476571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.598597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.442534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.316801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.278802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.401857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.410629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.262941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.526170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.814597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.985739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.054523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.880675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.868602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.977816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.175947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.510194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.691035] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5809.719625] LustreError: 171989:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5809.735371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.198693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.554453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.766093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.662308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.060610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.372560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.495237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.332644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.220124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.195242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.122372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.855616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.517058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.440554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.227707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.210975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.150388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.802939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.857085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.740579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.781770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.696867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.677696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.626521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.553836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.546584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.530360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.429182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.672849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.670230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.671967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.577344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.741724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.580456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.532866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.546189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.653442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.606786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.725367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.723805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.817329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.792872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.799069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.804682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.732804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.689129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.811848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.778610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.788667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.597228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.728198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.655638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.692156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.693525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.874067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.773836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.813042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.502138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.645682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.819627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.769909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.677972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.508207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.400868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.440115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.504785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.649490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.553784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.808119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.795895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.964749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.914258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.775831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.712599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.690756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.626843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.554483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.417976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.139506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.768626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.818521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.601881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.381453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.974094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.692574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.638804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.885646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.794814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.623603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.446520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.550873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.498648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.800506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.824822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.858638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.794909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.583730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.209992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.116907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.004462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.962594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.760866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.803128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.802915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.852686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.877552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.789832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.788469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.844715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.053155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.163952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.118031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.988333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.953888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.862959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.992133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.989614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.091309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.987543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.844912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.044493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.005496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.151993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.022542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.018682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.742688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.786871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.777971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.927941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.828883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.623257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.304307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.125971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.834668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.659940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.557883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.499850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.297337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.522410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.254343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.580300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.335742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.505274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.510923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.799873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.939690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.923325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.968222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.998763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.968618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.962852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.920971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.766987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.767526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.703924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.520303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.429279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.346122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.386067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.273259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.479395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.475979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.709607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.801392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.868820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.978478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.337531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.387382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.424086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.562299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.489655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.766925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.820817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.860886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.056939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.936564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.029923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.068652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.960549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.129454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.196536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.293386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.367960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.383417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.293297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.301390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.521830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.652288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.715903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.713907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.665633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.569734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.505492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.287401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.296590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.196384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.031169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.854740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.530960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.270908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.125444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.017488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.881309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.908547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.759661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.518755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.417748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.239756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.989377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.833142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.664763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.529085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.506272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.367824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.275409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.133639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.182055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.153663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.160847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.151313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.259978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.334131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.295651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.535842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.498835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.764672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.691812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.638756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.582860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.633793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.641349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.668256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.767238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.863602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.845069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.940369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.775570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.756921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.772523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.986771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.076043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.165781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.109867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.077015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.124805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.253772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.333795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.505648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.765439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.867843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.844939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.788149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.766051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.852456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.911246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.879505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.949161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.880224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.781672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.718252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.620776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.428239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.382486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.198166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.945385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.790986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.640945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.537768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.327395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.050917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.860647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.750431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.657302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.498599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.219812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.091034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.922246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.088123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.091169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.984771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.000760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.159288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.358754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.236181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.369581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.315538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.620874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.746120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.875527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.867490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.954846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.874975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.158611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.305337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.393869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.845663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.816258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.888490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.915265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.062680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.112838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.975847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.161592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.166921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.415237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.339490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.215872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.144690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.128150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.001693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.175332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.275592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.165470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.086861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.074483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.969514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.014102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.144578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.090859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.098305] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6125.119838] LustreError: 171989:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6125.136956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.203816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.330855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.215985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.325692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.336679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.408247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.396688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.467743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.408300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.350513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.218132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.027985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.968574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.752315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.490051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.307722] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6140.316078] LustreError: 171989:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6140.325573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.207554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.007615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.836876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.821642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.856575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.740091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.786107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.800669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.805674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.789972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.662224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.508705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.486614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.385867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.514900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.383561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.403707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.452753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.350665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.254732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.227704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.075961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.183570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.122961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.564041] LustreError: 173791:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6504.569922] LustreError: 171989:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7118.311899] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7118.315393] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7118.316423] LustreError: Skipped 1 previous similar message [ 7118.317154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7118.325277] Lustre: Skipped 471 previous similar messages [ 7118.347790] Lustre: Skipped 1 previous similar message [ 7122.289428] Lustre: server umount lustre-OST0000 complete [ 7128.546790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7128.553167] Lustre: Skipped 1 previous similar message [ 7132.189931] Lustre: server umount lustre-MDT0000 complete [ 7133.670532] LustreError: 173210:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7133.692166] LustreError: 173210:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7135.568563] LustreError: 171950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272182 with bad export cookie 6234079055110563438 [ 7135.572124] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7135.579770] LustreError: 171950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7135.947090] Lustre: server umount lustre-MDT0001 complete [ 7145.965576] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 7148.508785] Key type lgssc unregistered [ 7148.780626] LNet: 227225:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7148.794034] LNetError: 227225:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7148.832250] LNet: Removed LNI 192.168.204.144@tcp [ 7149.661178] Key type .llcrypt unregistered [ 7149.666165] Key type ._llcrypt unregistered [ 7176.067574] Key type ._llcrypt registered [ 7176.072758] Key type .llcrypt registered [ 7176.265360] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7177.531304] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7177.679573] alg: No test for adler32 (adler32-zlib) [ 7178.703428] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 7178.951568] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 7180.863201] Key type lgssc registered [ 7182.846577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7192.176124] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7192.228041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7193.756577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7198.295921] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7204.326233] LustreError: 228639:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7204.354881] LustreError: 228639:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7205.603570] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7205.993210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7209.811605] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7216.654564] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7222.183872] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7227.852133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7228.212908] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7233.547841] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7239.543804] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7240.860819] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7245.310068] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7247.293394] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7247.310505] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7247.324371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.339110] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7247.340072] LustreError: 228658:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7249.889193] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7249.903051] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7249.906694] Lustre: Skipped 1 previous similar message [ 7249.917566] Lustre: Skipped 1 previous similar message [ 7252.729605] Lustre: lustre-OST0000: Not available for connect from 192.168.204.44@tcp (stopping) [ 7255.017634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7255.034435] Lustre: Skipped 1 previous similar message [ 7255.783501] Lustre: server umount lustre-OST0000 complete [ 7267.343137] LustreError: 228658:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7289.540643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7289.843440] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7295.598120] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7302.447897] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7304.028682] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7306.212883] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.231684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.232526] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7309.623122] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9dba092ec800.ost_server_uuid 50 [ 7311.396121] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9dba092ec800.ost_server_uuid in FULL state after 0 sec [ 7316.450493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7316.452531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7316.459531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7316.466269] Lustre: Skipped 1 previous similar message [ 7321.574941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7321.584897] Lustre: Skipped 2 previous similar messages [ 7321.965974] Lustre: server umount lustre-OST0000 complete [ 7326.691086] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7331.809212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7331.820174] Lustre: Skipped 3 previous similar messages [ 7332.122486] Lustre: server umount lustre-MDT0000 complete [ 7336.077424] LustreError: 228620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272383 with bad export cookie 2438289754234516567 [ 7336.091137] LustreError: 228620:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7336.091890] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7336.491329] Lustre: server umount lustre-MDT0001 complete [ 7348.847947] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 7351.765851] Key type lgssc unregistered [ 7352.109739] LNet: 233847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7352.132785] LNetError: 233847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7353.201748] LNet: Removed LNI 192.168.204.144@tcp [ 7354.330273] Key type .llcrypt unregistered [ 7354.332437] Key type ._llcrypt unregistered [ 7385.777640] Key type ._llcrypt registered [ 7385.782675] Key type .llcrypt registered [ 7385.918946] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7386.795846] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7386.861039] alg: No test for adler32 (adler32-zlib) [ 7388.004819] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 7388.243867] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 7389.905924] Key type lgssc registered [ 7390.591946] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7398.642504] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7398.655057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7400.171796] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7405.071309] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7409.655587] LustreError: 235571:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7409.680446] LustreError: 235571:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7413.905601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7414.374791] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7419.227910] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7422.248975] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7429.280915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7429.782615] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7436.920743] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7445.189432] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7445.515072] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7445.660404] Lustre: lustre-OST0001: new disk, initializing [ 7445.662968] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7445.669211] Lustre: 237880:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7445.796203] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7448.172604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7448.187455] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7448.283559] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7452.774633] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7464.889945] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7470.063070] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7484.385756] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7484.398578] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7484.420218] Lustre: Skipped 2 previous similar messages [ 7484.440717] Lustre: Skipped 3 previous similar messages [ 7489.102629] Lustre: server umount lustre-MDT0000 complete [ 7489.512473] LustreError: 235566:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7489.546089] LustreError: 235566:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7494.442984] LustreError: 238650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272541 with bad export cookie 7519144436825627069 [ 7494.449789] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7494.452027] LustreError: 238650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7494.665691] Lustre: server umount lustre-MDT0001 complete [ 7508.829394] Lustre: server umount lustre-OST0000 complete [ 7511.007204] Lustre: 234834:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789272542/real 1789272542] req@ffff891de3af8380 x1876188132894592/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789272558 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7511.028375] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7511.038617] Lustre: Skipped 1 previous similar message [ 7512.530658] Lustre: server umount lustre-OST0001 complete [ 7531.562887] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7544.259312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7544.690805] LustreError: 240987:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7544.751303] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7549.926390] LustreError: 240988:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7550.205478] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7555.044266] LustreError: 240988:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7559.362952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7559.905378] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7564.194627] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7566.988033] Lustre: 242130:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7573.055590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7579.020330] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7584.632164] LustreError: 242481:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7586.898273] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7587.290294] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7587.294891] Lustre: Skipped 1 previous similar message [ 7590.389396] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7590.394309] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7594.074185] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7604.677681] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7608.911980] Lustre: 244002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7621.089110] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7621.096590] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7621.106077] Lustre: Skipped 3 previous similar messages [ 7621.110631] Lustre: Skipped 3 previous similar messages [ 7624.643214] Lustre: server umount mylustre-MDT0000 complete [ 7626.213097] LustreError: 240988:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7626.237972] LustreError: 240988:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7627.808345] LustreError: 240962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272675 with bad export cookie 7519144436825629876 [ 7627.813982] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7627.818027] LustreError: 240962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7628.080835] Lustre: server umount mylustre-MDT0001 complete [ 7640.774376] Lustre: server umount mylustre-OST0000 complete [ 7654.021545] Lustre: server umount mylustre-OST0001 complete [ 7679.125439] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7690.226663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7690.618387] LustreError: 246308:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7690.707468] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7695.034247] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7703.778600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7708.221323] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7710.897497] Lustre: 247451:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7718.201190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7726.015824] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7734.704993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7736.061658] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7736.065093] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7740.642509] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7748.112281] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7752.581636] Lustre: 249325:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7761.903621] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7761.920257] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7761.941235] Lustre: Skipped 3 previous similar messages [ 7761.944831] Lustre: Skipped 2 previous similar messages [ 7763.396149] Lustre: server umount tfs-MDT0000 complete [ 7767.019710] LustreError: 246304:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7767.055034] LustreError: 246304:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7767.195821] LustreError: 246284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272814 with bad export cookie 7519144436825631766 [ 7767.213488] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7767.741670] Lustre: server umount tfs-MDT0001 complete [ 7782.478537] Lustre: server umount tfs-OST0000 complete [ 7795.844503] Lustre: server umount tfs-OST0001 complete [ 7820.057557] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7830.773232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7831.291585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7831.299912] Lustre: Skipped 3 previous similar messages [ 7835.690677] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7843.668700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7848.487697] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7851.151658] Lustre: 252717:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7856.925098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7863.367667] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7872.224867] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7877.587633] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7877.600593] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7878.873347] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7887.747664] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7891.666265] Lustre: 254586:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7898.475056] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 00:15:45 (1789272945) [ 7916.000514] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7916.008642] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7916.028958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7916.036453] Lustre: Skipped 1 previous similar message [ 7920.672897] Lustre: server umount lustre-MDT0000 complete [ 7923.686612] LustreError: 251572:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7923.713369] LustreError: 251572:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7924.723496] LustreError: 251551:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789272972 with bad export cookie 7519144436825633432 [ 7924.731544] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7924.744376] LustreError: 251551:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7928.816225] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7928.822471] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7928.829785] Lustre: Skipped 4 previous similar messages [ 7928.850640] Lustre: Skipped 5 previous similar messages [ 7931.493061] Lustre: server umount lustre-MDT0001 complete [ 7950.303166] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7950.528340] Lustre: server umount lustre-OST0000 complete [ 7960.508414] Lustre: server umount lustre-OST0001 complete [ 7967.532124] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 7978.297181] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8014.007614] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8014.227805] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8014.269985] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8014.387573] Lustre: lustre-MDT0000: new disk, initializing [ 8014.521413] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8014.527912] Lustre: Skipped 3 previous similar messages [ 8014.562459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8019.323967] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8029.784818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8029.967075] Lustre: 257771:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8030.078344] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8030.088951] Lustre: Skipped 1 previous similar message [ 8030.231399] Lustre: lustre-MDT0001: new disk, initializing [ 8030.370389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8030.379164] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8034.751340] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8047.356906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8047.973920] Lustre: lustre-OST0000: new disk, initializing [ 8047.995359] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8047.999106] Lustre: 259354:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8049.499155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8049.510836] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8049.704601] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8055.792980] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8067.314043] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8067.575518] Lustre: lustre-OST0001: new disk, initializing [ 8067.586424] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8067.607785] Lustre: 260224:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8069.649739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8069.665442] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8069.763950] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8074.349287] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8089.608912] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 00:18:56 (1789273136) [ 8109.457508] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 00:19:15 (1789273155) [ 8125.930353] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8125.934280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8125.943865] Lustre: Skipped 1 previous similar message [ 8125.951623] Lustre: Skipped 3 previous similar messages [ 8140.257166] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8140.357605] Lustre: server umount lustre-OST0000 complete [ 8146.409315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8146.426370] Lustre: Skipped 6 previous similar messages [ 8150.610693] Lustre: server umount lustre-MDT0000 complete [ 8154.769822] LustreError: 260223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789273202 with bad export cookie 7519144436825634986 [ 8154.780269] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8154.795900] LustreError: 260223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8161.297695] Lustre: server umount lustre-MDT0001 complete [ 8178.626612] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 8256.781001] Lustre: server umount lustre-OST0001 complete [ 8265.112411] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 8284.726525] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8333.881808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8334.783874] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8334.948293] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8335.286332] Lustre: lustre-MDT0000: new disk, initializing [ 8335.567573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8335.587758] Lustre: Skipped 3 previous similar messages [ 8335.718060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8343.876768] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8357.826928] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8357.981213] Lustre: 265087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8358.069236] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8358.073636] Lustre: Skipped 1 previous similar message [ 8358.233822] Lustre: lustre-MDT0001: new disk, initializing [ 8358.600970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8358.615981] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8364.775612] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8377.031058] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8383.113325] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8390.782129] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8391.100362] Lustre: lustre-OST0000: new disk, initializing [ 8391.104771] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8391.115441] Lustre: 267058:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8393.182919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8393.200073] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8393.340109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8398.243489] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8409.685158] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8418.787409] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8418.802431] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8418.804031] LustreError: Skipped 1 previous similar message [ 8418.807074] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8418.807081] Lustre: Skipped 2 previous similar messages [ 8418.843658] Lustre: Skipped 5 previous similar messages [ 8423.901884] Lustre: server umount lustre-OST0000 complete [ 8437.063311] Lustre: server umount lustre-MDT0000 complete [ 8439.265130] LustreError: 265097:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8439.305126] LustreError: 265097:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 8441.888390] LustreError: 265078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789273489 with bad export cookie 7519144436825636456 [ 8441.898407] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8441.903394] LustreError: 265078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8442.601326] Lustre: server umount lustre-MDT0001 complete [ 8454.718551] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 8457.323553] Key type lgssc unregistered [ 8457.701651] LNet: 268820:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8457.715480] LNetError: 268820:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8457.746346] LNet: Removed LNI 192.168.204.144@tcp [ 8458.727281] Key type .llcrypt unregistered [ 8458.731922] Key type ._llcrypt unregistered [ 8494.612941] Key type ._llcrypt registered [ 8494.618304] Key type .llcrypt registered [ 8494.726680] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8495.463850] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8495.580146] alg: No test for adler32 (adler32-zlib) [ 8496.763330] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 8497.005958] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 8498.671400] Key type lgssc registered [ 8500.587745] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8511.276872] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8511.320266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8513.124742] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8518.320549] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8522.724163] LustreError: 270283:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8522.765159] LustreError: 270283:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8527.467620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8527.857412] LustreError: 270284:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8528.161457] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8533.669642] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8544.128418] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8551.268171] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8559.394676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8559.911969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8567.688791] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8576.696379] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8577.887741] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8584.422820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8584.433460] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8586.670839] Lustre: server umount lustre-OST0000 complete [ 8594.918720] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8594.932323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8594.933679] Lustre: Skipped 1 previous similar message [ 8594.934495] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8594.964526] Lustre: Skipped 2 previous similar messages [ 8596.905805] Lustre: server umount lustre-MDT0000 complete [ 8600.033544] LustreError: 270284:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8600.061298] LustreError: 270284:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8601.440497] LustreError: 272813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789273648 with bad export cookie 4929540400786280135 [ 8601.445370] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8601.453095] LustreError: 272813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8601.873364] Lustre: server umount lustre-MDT0001 complete [ 8615.710492] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 8619.843115] Key type lgssc unregistered [ 8620.304185] LNet: 273867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8620.310288] LNetError: 273867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8620.324743] LNet: Removed LNI 192.168.204.144@tcp [ 8621.160662] Key type .llcrypt unregistered [ 8621.167092] Key type ._llcrypt unregistered [ 8643.268269] Key type ._llcrypt registered [ 8643.270046] Key type .llcrypt registered [ 8643.374137] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8644.180968] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8644.199051] alg: No test for adler32 (adler32-zlib) [ 8645.121968] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 8645.256686] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 8646.896640] Key type lgssc registered [ 8648.551986] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8660.305201] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8660.360906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8675.999747] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8681.119504] LustreError: 275061:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.144@tcp: failed processing log, type 4: rc = -110 [ 8707.809273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8716.653566] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8725.969588] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8728.084727] Lustre: server umount lustre-OST0000 complete [ 8750.483510] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 8754.787252] Key type lgssc unregistered [ 8755.336801] LNet: 276692:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8755.357686] LNetError: 276692:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8755.382392] LNet: Removed LNI 192.168.204.144@tcp [ 8756.550540] Key type .llcrypt unregistered [ 8756.553173] Key type ._llcrypt unregistered [ 8811.504332] Key type ._llcrypt registered [ 8811.509103] Key type .llcrypt registered [ 8811.652287] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8812.986039] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8813.122042] alg: No test for adler32 (adler32-zlib) [ 8814.473635] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 8814.791497] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 8816.519415] Key type lgssc registered [ 8817.935058] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8832.082600] loop0: detected capacity change from 0 to 204800000 [ 8832.344535] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8832.371634] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8832.914805] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8834.239523] loop0: detected capacity change from 0 to 204800000 [ 8834.479510] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8834.499610] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8834.967711] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8836.256264] loop0: detected capacity change from 0 to 204800000 [ 8836.515755] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8836.528301] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8836.661832] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8837.989820] loop0: detected capacity change from 0 to 204800000 [ 8838.257643] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8838.271160] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8838.469203] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8839.800711] loop0: detected capacity change from 0 to 204800000 [ 8839.832700] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8845.014990] loop0: detected capacity change from 0 to 204800000 [ 8845.082147] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8848.919966] loop0: detected capacity change from 0 to 204800000 [ 8849.007386] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8853.114816] loop0: detected capacity change from 0 to 204800000 [ 8853.166856] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8857.140417] loop0: detected capacity change from 0 to 204800000 [ 8857.202707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8857.245489] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8857.317344] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8857.323680] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8857.343107] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8857.349335] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8857.356969] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8857.365974] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8857.372570] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8857.379822] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8857.386942] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8857.397551] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8857.411176] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8857.421145] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8857.436268] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8857.444592] Lustre: 280109:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8862.863903] Lustre: server umount MGS complete [ 8863.592956] loop0: detected capacity change from 0 to 204800000 [ 8863.654080] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8864.273124] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8864.313080] Lustre: lustre-MDT0000: Aborting client recovery [ 8865.222466] loop1: detected capacity change from 0 to 204800000 [ 8865.340146] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8865.466951] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8865.481680] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8865.495229] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8865.512033] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8865.522336] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8865.547302] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8865.562152] Lustre: 280541:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8865.589928] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8865.791755] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8865.893609] Lustre: lustre-MDT0001: Aborting client recovery [ 8866.553116] loop2: detected capacity change from 0 to 204800000 [ 8866.681719] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8866.756216] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8866.766820] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8866.772910] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8866.781961] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8866.792055] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8866.805709] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8866.815466] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8866.917091] Lustre: Found index 0 for lustre-OST0000, updating log [ 8866.935460] Lustre: 280639:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8867.029516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8867.069953] Lustre: lustre-OST0000: Aborting recovery [ 8867.918781] loop3: detected capacity change from 0 to 204800000 [ 8868.014575] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8868.077802] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8868.087238] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8868.093774] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8868.103533] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8868.108935] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8868.115913] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8868.121737] Lustre: 280704:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8868.134484] Lustre: Found index 1 for lustre-OST0001, updating log [ 8868.154654] Lustre: 280710:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8868.232810] Lustre: lustre-OST0001: Aborting recovery [ 8868.983859] Lustre: lustre-MDT0000: trigger OI scrub by RPC for [0x200000402:0x2:0x0]/0 with flags 0x4a: rc = 0 [ 8873.500297] LustreError: 280485:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8874.476701] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8874.564451] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8874.608593] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8874.639485] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8876.546978] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8876.548078] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8876.936506] LustreError: 280451:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 969us: rc = -2 [ 8877.440778] LustreError: 280450:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 940us: rc = -2 [ 8877.466585] LustreError: 280450:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 8891.361390] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8891.375180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8891.394311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8891.877538] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8891.903430] Lustre: Skipped 2 previous similar messages [ 8891.909252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8891.912084] Lustre: Skipped 2 previous similar messages [ 8897.004817] LustreError: 280454:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8897.048809] LustreError: 280454:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8897.148228] Lustre: server umount lustre-MDT0000 complete [ 8898.044709] LustreError: 280434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789273945 with bad export cookie 11619687150912976265 [ 8898.046122] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8898.055055] LustreError: 280434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8902.111907] LustreError: 280450:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8902.112451] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8902.145481] LustreError: 280450:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8902.169502] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8902.203845] Lustre: Skipped 1 previous similar message [ 8903.136560] LustreError: 280947:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8903.151322] LustreError: 280947:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8904.526469] Lustre: server umount lustre-MDT0001 complete [ 8920.031197] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8920.178466] Lustre: server umount lustre-OST0000 complete [ 8927.777893] Lustre: server umount lustre-OST0001 complete [ 8936.648831] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 00:33:03 (1789273983) [ 8968.325953] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 8979.771670] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 9017.499328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9017.684519] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9017.717981] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9017.806983] Lustre: lustre-MDT0000: new disk, initializing [ 9017.885910] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9017.889845] Lustre: Skipped 1 previous similar message [ 9017.908187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9021.715410] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9031.605850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9031.768732] Lustre: 284779:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9031.841903] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9031.851347] Lustre: Skipped 1 previous similar message [ 9031.934692] Lustre: lustre-MDT0001: new disk, initializing [ 9032.090968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9032.135858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9032.146674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9036.638873] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9046.647283] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9055.335706] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9064.637549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9065.027950] Lustre: lustre-OST0000: new disk, initializing [ 9065.030834] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9065.035974] Lustre: 286750:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9065.201564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9070.639246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9070.645640] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9070.748083] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9073.304717] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9083.553988] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9089.701803] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9099.249264] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9107.074692] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9117.172961] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9157.601344] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9157.609486] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9157.622483] Lustre: Skipped 1 previous similar message [ 9157.630325] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9157.636109] Lustre: Skipped 2 previous similar messages [ 9163.241626] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9163.252382] Lustre: Skipped 3 previous similar messages [ 9168.363600] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9168.696908] Lustre: server umount lustre-OST0000 complete [ 9173.473424] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9173.480807] Lustre: Skipped 2 previous similar messages [ 9173.482577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9173.490775] Lustre: Skipped 3 previous similar messages [ 9186.271159] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9186.580737] Lustre: server umount lustre-MDT0000 complete [ 9188.840068] LustreError: 284792:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9188.857439] LustreError: 284792:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9189.908291] LustreError: 285585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274237 with bad export cookie 11619687150912978974 [ 9189.912566] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9189.930459] LustreError: 285585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9190.322468] Lustre: server umount lustre-MDT0001 complete [ 9200.362782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9205.128421] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9209.857731] Lustre: server umount MGS complete [ 9221.073406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9221.675961] LustreError: 290624:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9221.697812] LustreError: 290637:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9221.769119] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9228.048380] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9232.355056] LustreError: 290625:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9232.380955] LustreError: 290625:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9239.715320] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9246.098873] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9254.134766] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9259.344348] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9265.721107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9266.238470] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9266.248248] Lustre: Skipped 1 previous similar message [ 9271.810501] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9280.870485] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9308.129501] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9308.144156] LustreError: Skipped 1 previous similar message [ 9308.157452] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9308.169896] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9308.174481] Lustre: Skipped 4 previous similar messages [ 9309.133919] Lustre: server umount lustre-OST0000 complete [ 9319.519564] Lustre: server umount lustre-MDT0000 complete [ 9321.440240] LustreError: 290624:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9321.475584] LustreError: 290624:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9323.916818] LustreError: 290605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274371 with bad export cookie 11619687150912980913 [ 9323.923563] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9323.933476] LustreError: 290605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9334.465494] Lustre: server umount lustre-MDT0001 complete [ 9345.885081] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 9349.209555] Key type lgssc unregistered [ 9349.534574] LNet: 294569:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9349.548225] LNetError: 294569:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9349.567993] LNet: Removed LNI 192.168.204.144@tcp [ 9350.384980] Key type .llcrypt unregistered [ 9350.387253] Key type ._llcrypt unregistered [ 9387.711597] Key type ._llcrypt registered [ 9387.716788] Key type .llcrypt registered [ 9387.920547] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 9404.550688] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 9405.754423] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9405.868024] alg: No test for adler32 (adler32-zlib) [ 9407.002596] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 9407.222532] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 9408.951713] Key type lgssc registered [ 9410.112891] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9449.367669] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9449.387939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9450.828146] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9450.904328] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9451.052064] Lustre: lustre-MDT0000: new disk, initializing [ 9451.180236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9451.207848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9455.930876] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9468.229787] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9468.391675] Lustre: 298521:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9468.448869] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9468.455508] Lustre: Skipped 1 previous similar message [ 9468.586187] Lustre: lustre-MDT0001: new disk, initializing [ 9468.812268] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9468.908310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9468.929165] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9473.839443] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9484.685414] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9489.684986] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9497.142674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9497.385446] Lustre: lustre-OST0000: new disk, initializing [ 9497.388738] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9497.392939] Lustre: 300491:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9497.463845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9503.450842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9503.465710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9503.579514] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9504.487298] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9514.845644] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9522.160991] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9533.735821] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9542.015090] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9550.926876] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9595.881760] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9595.881844] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9595.910776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9600.996183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9601.010212] Lustre: Skipped 2 previous similar messages [ 9606.130807] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9606.137700] Lustre: Skipped 1 previous similar message [ 9610.208913] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9610.560200] Lustre: server umount lustre-OST0000 complete [ 9616.352501] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9616.361304] Lustre: Skipped 2 previous similar messages [ 9616.361819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9616.376302] Lustre: Skipped 1 previous similar message [ 9620.631725] Lustre: server umount lustre-MDT0000 complete [ 9624.545716] LustreError: 298515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274671 with bad export cookie 3091759651302206139 [ 9624.556243] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9624.570922] LustreError: 298515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9624.822374] Lustre: server umount lustre-MDT0001 complete [ 9633.069521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9638.004908] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9643.466644] Lustre: server umount MGS complete [ 9656.017753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9656.404263] LustreError: 304416:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9656.431178] LustreError: 304416:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9656.449846] LustreError: 304429:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9656.522854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9661.924480] LustreError: 304417:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9662.158835] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9666.019647] LustreError: 304416:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9671.137104] LustreError: 304417:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9672.522112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9673.165587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9679.507674] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9687.602613] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9694.578088] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9702.247543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9702.635768] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9709.248190] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9717.649633] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9762.272992] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9762.290945] LustreError: Skipped 1 previous similar message [ 9762.294487] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9762.306863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9762.344794] Lustre: Skipped 1 previous similar message [ 9762.373567] Lustre: Skipped 1 previous similar message [ 9768.374070] Lustre: server umount lustre-OST0000 complete [ 9772.513145] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9772.517296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9772.530429] Lustre: Skipped 1 previous similar message [ 9772.547922] Lustre: Skipped 3 previous similar messages [ 9778.618971] Lustre: server umount lustre-MDT0000 complete [ 9782.322567] LustreError: 304397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789274829 with bad export cookie 3091759651302208099 [ 9782.323703] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9782.346648] LustreError: 304397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9782.698258] Lustre: server umount lustre-MDT0001 complete [ 9793.287952] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 9796.751474] Key type lgssc unregistered [ 9797.189229] LNet: 308383:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9797.205140] LNetError: 308383:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9797.225886] LNet: Removed LNI 192.168.204.144@tcp [ 9797.981492] Key type .llcrypt unregistered [ 9797.984894] Key type ._llcrypt unregistered [ 9821.540830] Key type ._llcrypt registered [ 9821.543450] Key type .llcrypt registered [ 9821.650270] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 9834.645402] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 9836.395391] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9836.593326] alg: No test for adler32 (adler32-zlib) [ 9837.625395] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [ 9837.872697] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 9839.583196] Key type lgssc registered [ 9840.776219] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9866.699508] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 9881.263445] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9881.307865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9882.624516] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9882.675187] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9882.826220] Lustre: lustre-MDT0000: new disk, initializing [ 9882.948291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9883.003794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9888.529466] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9900.095860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9900.322604] Lustre: lustre-OST0000: new disk, initializing [ 9900.325910] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9900.330935] Lustre: Skipped 1 previous similar message [ 9900.335680] Lustre: 312412:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9900.412981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9907.960963] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 9909.413423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9909.424986] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9909.485991] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10790.370366] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10790.392987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10795.458786] Lustre: server umount lustre-MDT0000 complete [10798.508263] LustreError: 311595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789275845 with bad export cookie 16326770521063826944 [10798.517140] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10798.525094] LustreError: 311595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10798.747732] Lustre: server umount lustre-OST0000 complete [10823.269581] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [10834.567422] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [10874.947924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10875.225970] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10875.278627] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10875.422874] Lustre: lustre-MDT0000: new disk, initializing [10875.632847] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10875.693379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10880.444701] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [10890.889235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10890.965550] Lustre: 316767:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10891.003735] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10891.011383] Lustre: Skipped 1 previous similar message [10891.111386] Lustre: lustre-MDT0001: new disk, initializing [10891.302521] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10891.373855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10891.395103] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10896.009489] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [10905.838673] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10911.436180] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10919.315666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10919.643258] Lustre: lustre-OST0000: new disk, initializing [10919.647490] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10919.656192] Lustre: 318738:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10919.748647] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10926.585373] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [10929.225770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10929.250502] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10929.350592] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10936.300665] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10941.535754] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10941.814266] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10946.425631] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10946.661726] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10950.123483] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10950.135751] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10950.153748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10950.163929] LustreError: Skipped 1 previous similar message [10950.182239] Lustre: Skipped 1 previous similar message [10954.944732] Lustre: server umount lustre-OST0000 complete [10960.355973] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10960.369381] Lustre: Skipped 1 previous similar message [10960.382317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10960.387205] Lustre: Skipped 1 previous similar message [10965.472970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10966.166337] Lustre: server umount lustre-MDT0000 complete [10970.766474] LustreError: 316759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276018 with bad export cookie 16326770521066767763 [10970.769996] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10970.782976] LustreError: 316759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10971.243254] Lustre: server umount lustre-MDT0001 complete [10974.573842] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10976.316348] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 01:07:03 (1789276023) [10983.171236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10983.810361] LustreError: 321048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10983.837485] LustreError: 321048:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10983.933459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10988.212721] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [10989.028409] LustreError: 321049:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10994.152137] LustreError: 321048:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10997.164608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10997.652060] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11002.467503] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11009.859934] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11015.952446] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11023.009749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11023.378487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11029.950479] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11037.810706] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11044.811340] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11044.954617] Lustre: lustre-OST0001: new disk, initializing [11044.957315] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11044.961483] Lustre: 323885:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11045.067354] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11051.075507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11051.086970] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11051.133918] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11051.146367] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11052.206974] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11052.237557] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11053.284315] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11053.294238] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11055.343366] LustreError: 324246:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11055.376511] LustreError: 324246:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11059.493514] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11059.520944] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11063.574651] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11063.987686] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11067.810568] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11067.833069] LustreError: 324234:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11070.955955] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9dba3bd44000.ost_server_uuid 50 [11073.114604] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9dba3bd44000.ost_server_uuid in FULL state after 0 sec [11079.802935] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9dba3bd44000.ost_server_uuid 50 [11081.897275] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9dba3bd44000.ost_server_uuid in IDLE state after 0 sec [11084.261292] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11099.619163] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11099.621821] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11099.627453] LustreError: Skipped 1 previous similar message [11099.651772] Lustre: Skipped 1 previous similar message [11099.652460] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11099.670394] Lustre: Skipped 2 previous similar messages [11104.512660] Lustre: server umount lustre-OST0001 complete [11107.537673] LustreError: 322870:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11111.410996] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11111.425298] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11111.450035] Lustre: Skipped 1 previous similar message [11111.467861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11111.477155] Lustre: Skipped 3 previous similar messages [11112.947574] LustreError: 322871:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11112.998026] LustreError: 322871:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11116.757767] Lustre: server umount lustre-OST0000 complete [11123.171232] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11123.195431] Lustre: Skipped 1 previous similar message [11128.288582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11128.308181] Lustre: Skipped 5 previous similar messages [11128.644222] Lustre: server umount lustre-MDT0000 complete [11132.713255] LustreError: 321031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276180 with bad export cookie 16326770521066768645 [11132.718105] LustreError: 321031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11132.724807] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11133.253474] Lustre: server umount lustre-MDT0001 complete [11144.631146] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [11148.236950] Key type lgssc unregistered [11148.490057] LNet: 326572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11148.502324] LNetError: 326572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11148.530280] LNet: Removed LNI 192.168.204.144@tcp [11149.854870] Key type .llcrypt unregistered [11149.866764] Key type ._llcrypt unregistered [11176.362784] Key type ._llcrypt registered [11176.368141] Key type .llcrypt registered [11176.558153] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [11177.522745] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11177.547405] alg: No test for adler32 (adler32-zlib) [11178.687488] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [11178.988793] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [11180.703198] Key type lgssc registered [11181.961249] Lustre: Echo OBD driver; http://www.lustre.org/ [11190.481970] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11190.501566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11192.115144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11197.482827] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11202.530729] LustreError: 327990:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11202.554870] LustreError: 327990:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11205.946135] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11207.651012] LustreError: 327991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11212.770524] LustreError: 327990:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11213.649379] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11214.120289] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11219.221345] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11229.002294] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11229.559305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11236.709032] LustreError: 329668:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11236.762192] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11236.848985] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11244.004774] LustreError: 329670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11244.050862] LustreError: 329670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11245.279965] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11252.837386] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11253.235616] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11258.387078] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11261.308282] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11270.716428] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11278.437298] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11278.649285] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11386.339585] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11386.367793] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11386.374032] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11386.382901] Lustre: Skipped 1 previous similar message [11391.462987] LustreError: 330893:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11391.485473] LustreError: 330893:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11391.507413] Lustre: server umount lustre-OST0001 complete [11400.673062] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11400.683719] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11400.701549] Lustre: Skipped 1 previous similar message [11400.717270] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11401.714542] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11401.732336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11401.736870] Lustre: Skipped 1 previous similar message [11404.926387] Lustre: server umount lustre-OST0000 complete [11411.938617] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11411.943293] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11411.961362] Lustre: Skipped 1 previous similar message [11411.983592] Lustre: Skipped 1 previous similar message [11415.473529] Lustre: server umount lustre-MDT0000 complete [11417.060478] LustreError: 331670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11417.081780] LustreError: 331670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11420.178078] LustreError: 331676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276467 with bad export cookie 15172179293471271122 [11420.189487] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11420.192351] LustreError: 331676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11420.621923] Lustre: server umount lustre-MDT0001 complete [11433.708148] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [11437.127517] Key type lgssc unregistered [11437.491900] LNet: 332936:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11437.518743] LNetError: 332936:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11437.544562] LNet: Removed LNI 192.168.204.144@tcp [11438.658091] Key type .llcrypt unregistered [11438.663102] Key type ._llcrypt unregistered [11491.179733] Key type ._llcrypt registered [11491.182088] Key type .llcrypt registered [11491.329787] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [11492.573436] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11492.624213] alg: No test for adler32 (adler32-zlib) [11493.946451] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [11494.226046] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [11495.976066] Key type lgssc registered [11497.433330] Lustre: Echo OBD driver; http://www.lustre.org/ [11508.491436] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11508.532133] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11510.073484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11515.266892] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11520.482278] LustreError: 335227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11520.511387] LustreError: 335227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11524.243619] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11524.616087] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11530.128634] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11533.248098] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11540.711756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11541.213722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11548.470224] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11549.422370] LustreError: 337119:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11554.804677] LustreError: 336717:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11555.832666] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11558.639846] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11558.935714] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11564.021070] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11565.848961] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11574.592791] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11585.043473] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11599.843825] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11599.855478] Lustre: Skipped 3 previous similar messages [11599.857586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11599.867461] Lustre: Skipped 3 previous similar messages [11603.387860] Lustre: server umount lustre-MDT0000 complete [11604.978017] LustreError: 335227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11605.015509] LustreError: 335227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [11607.502422] LustreError: 335205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276654 with bad export cookie 650617003283747317 [11607.506285] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11607.520097] LustreError: 335205:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11608.081375] Lustre: server umount lustre-MDT0001 complete [11621.763778] Lustre: server umount lustre-OST0000 complete [11636.249670] Lustre: server umount lustre-OST0001 complete [11655.694437] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [11667.639525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11668.134906] LustreError: 340490:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11668.251801] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11673.485408] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11677.665792] LustreError: 340490:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11677.704942] LustreError: 340490:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11683.297730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11684.002363] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11689.415853] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11693.103146] Lustre: 341633:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11700.704807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11701.092524] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11708.773877] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11709.305172] LustreError: 342372:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11709.331157] LustreError: 342372:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11713.473955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11719.216440] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11724.800969] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11725.862919] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11734.220388] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11737.922776] Lustre: 343504:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11745.251629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11745.256488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11745.268102] Lustre: Skipped 1 previous similar message [11745.282200] Lustre: Skipped 3 previous similar messages [11748.260668] Lustre: server umount lustre-MDT0000 complete [11750.373373] LustreError: 341223:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11750.401282] LustreError: 341223:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [11752.528464] LustreError: 341235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789276799 with bad export cookie 650617003283749018 [11752.532270] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11752.551987] LustreError: 341235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11753.150707] Lustre: server umount lustre-MDT0001 complete [11768.664494] Lustre: server umount lustre-OST0000 complete [11771.492359] Lustre: 334489:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789276802/real 1789276802] req@ffff891dd0fcdf80 x1876192438374400/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789276818 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11771.526430] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11771.537068] Lustre: Skipped 2 previous similar messages [11773.042748] Lustre: server umount lustre-OST0001 complete [11777.464733] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11779.270527] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11781.264195] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 01:20:28 (1789276828) [11807.965829] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [11818.776582] loop0: detected capacity change from 0 to 1099511627776 [11819.555884] SGI XFS with ACLs, security attributes, quota, no debug enabled [11819.565057] *** VALIDATE xfs *** [11819.621749] XFS (loop0): Mounting V5 Filesystem [11819.654985] XFS (loop0): Ending clean mount [11821.314369] loop1: detected capacity change from 0 to 18691697672192 [11921.840572] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [11934.208417] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [11976.508354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11976.808221] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11976.836578] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11976.919572] Lustre: lustre-MDT0000: new disk, initializing [11976.991183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11977.001667] Lustre: Skipped 1 previous similar message [11977.021632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11982.245777] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [11993.649691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11993.792732] Lustre: 348642:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11993.912637] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11993.916693] Lustre: Skipped 1 previous similar message [11994.026264] Lustre: lustre-MDT0001: new disk, initializing [11994.229299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11994.239348] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11999.570974] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12010.001578] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12016.593336] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12023.923467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12024.317777] Lustre: lustre-OST0000: new disk, initializing [12024.329630] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12024.344782] Lustre: 350610:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12026.402962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12026.412451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12026.529261] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12030.857957] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12040.936509] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12046.371979] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12046.660622] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12052.401746] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12052.730216] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12057.579859] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12057.584326] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12057.592456] LustreError: Skipped 1 previous similar message [12057.594435] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12057.659404] Lustre: Skipped 1 previous similar message [12062.692164] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12062.718361] Lustre: Skipped 2 previous similar messages [12067.811284] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12067.829876] Lustre: Skipped 1 previous similar message [12069.344596] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12069.699780] Lustre: server umount lustre-OST0000 complete [12076.001436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12076.011794] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12076.044834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12080.032258] Lustre: server umount lustre-MDT0000 complete [12083.171237] LustreError: 348653:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12083.201438] LustreError: 348653:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12084.121457] LustreError: 348632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789277131 with bad export cookie 650617003283750565 [12084.129567] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12084.134195] LustreError: 348632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12084.426886] Lustre: server umount lustre-MDT0001 complete [12092.899516] XFS (loop0): Unmounting Filesystem [12098.039247] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 01:25:44 (1789277144) [12105.073054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12105.842271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12105.862902] Lustre: Skipped 2 previous similar messages [12110.153689] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12118.515664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12122.746895] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12130.332970] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12135.960954] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12143.289296] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12149.461345] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12155.843728] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12165.090246] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12165.097448] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12165.110131] Lustre: Skipped 1 previous similar message [12165.115580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12165.117420] Lustre: Skipped 2 previous similar messages [12168.618186] Lustre: server umount lustre-OST0000 complete [12175.330237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12175.343609] LustreError: Skipped 1 previous similar message [12188.127128] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12188.404554] Lustre: server umount lustre-MDT0000 complete [12192.321956] LustreError: 353152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789277239 with bad export cookie 650617003283751447 [12192.328872] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12192.335184] LustreError: 353152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12192.730410] Lustre: server umount lustre-MDT0001 complete [12205.204622] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [12208.623471] Key type lgssc unregistered [12209.065957] LNet: 356801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12209.072743] LNetError: 356801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12209.086141] LNet: Removed LNI 192.168.204.144@tcp [12210.081208] Key type .llcrypt unregistered [12210.082729] Key type ._llcrypt unregistered [12237.415564] Key type ._llcrypt registered [12237.418489] Key type .llcrypt registered [12237.574711] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [12238.283948] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12238.333034] alg: No test for adler32 (adler32-zlib) [12239.309603] Lustre: Lustre: Build Version: 2.17.57_106_gd81f50d [12239.512322] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [12241.184215] Key type lgssc registered [12242.176316] Lustre: Echo OBD driver; http://www.lustre.org/ [12251.465096] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12251.508837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12253.485120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12257.932309] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12262.884239] LustreError: 358371:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12262.906818] LustreError: 358371:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12267.550668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12268.028334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12272.430154] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12279.531851] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12285.839750] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12293.955171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12294.526853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12301.194071] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12308.514971] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12319.205471] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12319.211711] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12319.232151] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12319.240990] Lustre: Skipped 1 previous similar message [12321.056304] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.44@tcp (stopping) [12324.334222] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12324.349034] Lustre: Skipped 1 previous similar message [12325.570937] Lustre: server umount lustre-MDT0001 complete [12326.098248] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12329.456414] LustreError: 360675:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12329.490496] LustreError: 360675:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12334.563492] LustreError: 358372:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12334.597910] LustreError: 358372:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12335.927912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12336.073887] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12336.093238] Lustre: Found index 1 for lustre-MDT0001, updating log [12336.157876] Lustre: 358361:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12336.431954] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12341.416212] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12341.740972] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12343.304395] LustreError: 361922:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12343.321185] LustreError: 361922:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12343.334131] LustreError: 358363:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12343.467730] LustreError: 361934:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.144@tcp: cfg command failed: rc = -17 [12343.479651] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12343.479651] [12351.751055] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.44@tcp (stopping) [12351.987441] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12352.011590] Lustre: Skipped 2 previous similar messages [12352.033509] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12355.808709] Lustre: server umount lustre-MDT0001 complete [12356.820052] LustreError: 358367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12366.980962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12367.063083] LustreError: 358368:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12367.080103] LustreError: 358368:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12367.122235] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12367.127603] Lustre: Found index 1 for lustre-MDT0001, updating log [12367.179182] Lustre: 358361:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12367.328868] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12367.337596] Lustre: Skipped 3 previous similar messages [12367.689916] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12372.985522] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12373.020166] LustreError: 362850:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12373.034951] LustreError: 362850:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12373.047183] LustreError: 358363:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12373.055809] LustreError: 358363:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12373.111605] LustreError: 362858:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12373.120680] LustreError: 362858:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12373.130579] LustreError: 362858:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.144@tcp: cfg command failed: rc = -17 [12373.142042] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.144@tcp [12373.142042] [12373.533617] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12680.161638] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12680.191989] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12680.215277] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12680.230906] Lustre: Skipped 1 previous similar message [12686.110306] Lustre: server umount lustre-MDT0001 complete [12689.617845] LustreError: 360675:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12696.458364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12696.574854] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12696.581540] Lustre: Found index 1 for lustre-MDT0001, updating log [12696.623377] Lustre: 358360:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12696.855821] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12701.234857] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [12702.189547] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12706.847801] LustreError: 371473:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12706.862588] LustreError: 371473:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12706.881028] LustreError: 358363:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12706.902179] LustreError: 358363:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12707.054811] LustreError: 371479:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12707.091866] LustreError: 371479:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.144@tcp: cfg command failed: rc = -114 [12707.114109] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12707.114109] [13029.858453] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13029.862937] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13029.863653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13029.863658] Lustre: Skipped 3 previous similar messages [13029.881029] Lustre: Skipped 2 previous similar messages [13031.643676] Lustre: server umount lustre-MDT0000 complete [13034.981672] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13035.001241] LustreError: 358366:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [13036.086337] LustreError: 362630:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789278083 with bad export cookie 13370738309000639370 [13036.088668] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13036.096149] LustreError: 362630:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13040.107236] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13040.121354] Lustre: Skipped 1 previous similar message [13042.676523] Lustre: server umount lustre-MDT0001 complete [13047.020090] Lustre: server umount lustre-OST0000 complete [13055.946951] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [13066.009283] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [13098.746761] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:42:26 (1789278146) [13106.036714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13106.288159] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13106.322284] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13106.421830] Lustre: lustre-MDT0000: new disk, initializing [13106.521708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13106.536672] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13111.790562] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [13122.893953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13123.059819] Lustre: 382154:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13123.117774] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13123.121632] Lustre: Skipped 1 previous similar message [13123.195220] Lustre: lustre-MDT0001: new disk, initializing [13123.299585] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13123.343094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13123.356934] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13128.319828] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [13138.751389] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13144.876374] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13151.900340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13152.284351] Lustre: lustre-OST0000: new disk, initializing [13152.291481] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13152.308944] Lustre: 384125:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13152.411986] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13159.542694] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [13161.079357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13161.097901] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13161.312596] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13170.500026] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13178.857628] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13178.866771] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13178.885126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13178.897476] Lustre: Skipped 4 previous similar messages [13179.377476] Lustre: server umount lustre-MDT0000 complete [13181.930291] LustreError: 382162:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13181.951818] LustreError: 382162:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [13183.377944] LustreError: 384124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789278230 with bad export cookie 13370738309000641330 [13183.380887] LustreError: 382961:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13183.384809] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13183.406142] LustreError: 384124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13183.749373] Lustre: server umount lustre-MDT0001 complete [13198.546851] Lustre: server umount lustre-OST0000 complete [13231.177330] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [13241.328895] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [13279.165543] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12996 sec ========= 01:45:25 (1789278325) [13280.972406] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 01:45:27 (1789278327) === [13284.628382] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 01:45:31 (1789278331) === [13325.607948] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [13328.802936] Key type lgssc unregistered [13329.151664] LNet: 390490:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13329.164836] LNetError: 390490:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13329.181304] LNet: Removed LNI 192.168.204.144@tcp [13330.177168] Key type .llcrypt unregistered [13330.178512] Key type ._llcrypt unregistered