[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 462888486 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003226] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009009] pid_max: default: 32768 minimum: 301 [ 0.010118] LSM: Security Framework initializing [ 0.011039] Yama: becoming mindful. [ 0.012042] SELinux: Initializing. [ 0.013075] *** VALIDATE selinux *** [ 0.022040] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027637] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028159] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029104] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030102] *** VALIDATE tmpfs *** [ 0.032102] *** VALIDATE proc *** [ 0.033247] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.036259] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037117] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039029] Spectre V2 : User space: Vulnerable [ 0.040008] Speculative Store Bypass: Vulnerable [ 0.042779] debug: unmapping init [mem 0xffffffffaf659000-0xffffffffaf660fff] [ 0.044150] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045762] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046026] ... version: 2 [ 0.047010] ... bit width: 48 [ 0.048013] ... generic registers: 4 [ 0.049014] ... value mask: 0000ffffffffffff [ 0.050016] ... max period: 00007fffffffffff [ 0.051015] ... fixed-purpose events: 3 [ 0.052013] ... event mask: 000000070000000f [ 0.053333] rcu: Hierarchical SRCU implementation. [ 0.055437] smp: Bringing up secondary CPUs ... [ 0.056619] x86: Booting SMP configuration: [ 0.057034] .... node #0, CPUs: #1 #2 #3 [ 0.061222] smp: Brought up 1 node, 4 CPUs [ 0.063021] smpboot: Max logical packages: 1 [ 0.064015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.187137] node 0 deferred pages initialised in 119ms [ 0.191022] devtmpfs: initialized [ 0.192293] x86/mm: Memory block size: 128MB [ 0.194029] gcov: version magic: 0x41383552 [ 0.197389] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.199144] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.200342] pinctrl core: initialized pinctrl subsystem [ 0.201233] [ 0.201704] ************************************************************* [ 0.202016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.203013] ** ** [ 0.204016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.205021] ** ** [ 0.206015] ** This means that this kernel is built to expose internal ** [ 0.207017] ** IOMMU data structures, which may compromise security on ** [ 0.208017] ** your system. ** [ 0.209017] ** ** [ 0.210021] ** If you see this message and you are not debugging the ** [ 0.211017] ** kernel, report this immediately to your vendor! ** [ 0.212015] ** ** [ 0.213028] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.214014] ************************************************************* [ 0.216000] NET: Registered protocol family 16 [ 0.217511] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.220085] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.223079] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.227143] cpuidle: using governor menu [ 0.230017] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.232900] PCI: Using configuration type 1 for base access [ 0.235154] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.244219] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.247066] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.251051] cryptd: max_cpu_qlen set to 1000 [ 0.253230] ACPI: Added _OSI(Module Device) [ 0.255022] ACPI: Added _OSI(Processor Device) [ 0.256015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.258018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.262265] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.269310] ACPI: Interpreter enabled [ 0.270083] ACPI: PM: (supports S0 S3 S4 S5) [ 0.272017] ACPI: Using IOAPIC for interrupt routing [ 0.273143] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.276373] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.285000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.287055] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.290025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.293092] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.297266] acpiphp: Slot [2] registered [ 0.298142] acpiphp: Slot [5] registered [ 0.299107] acpiphp: Slot [6] registered [ 0.300133] acpiphp: Slot [7] registered [ 0.301117] acpiphp: Slot [8] registered [ 0.302152] acpiphp: Slot [9] registered [ 0.303116] acpiphp: Slot [10] registered [ 0.304092] acpiphp: Slot [3] registered [ 0.304998] acpiphp: Slot [4] registered [ 0.306085] acpiphp: Slot [11] registered [ 0.307076] acpiphp: Slot [12] registered [ 0.308131] acpiphp: Slot [13] registered [ 0.310088] acpiphp: Slot [14] registered [ 0.311011] acpiphp: Slot [15] registered [ 0.312051] acpiphp: Slot [16] registered [ 0.313030] acpiphp: Slot [17] registered [ 0.314019] acpiphp: Slot [18] registered [ 0.315073] acpiphp: Slot [19] registered [ 0.316056] acpiphp: Slot [20] registered [ 0.316968] acpiphp: Slot [21] registered [ 0.317085] acpiphp: Slot [22] registered [ 0.317997] acpiphp: Slot [23] registered [ 0.319068] acpiphp: Slot [24] registered [ 0.320020] acpiphp: Slot [25] registered [ 0.320939] acpiphp: Slot [26] registered [ 0.322068] acpiphp: Slot [27] registered [ 0.323065] acpiphp: Slot [28] registered [ 0.323989] acpiphp: Slot [29] registered [ 0.325058] acpiphp: Slot [30] registered [ 0.326016] acpiphp: Slot [31] registered [ 0.327053] PCI host bridge to bus 0000:00 [ 0.328015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.329017] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.330019] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.332020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.334021] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.335039] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.337201] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.340206] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.343000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.350017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.356017] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.358019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.359011] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.361016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.362647] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.364596] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.367068] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.370947] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.375016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.387914] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.392017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.402452] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.411021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.419021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.434027] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.444186] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.452026] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.459026] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.474028] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.485525] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.494021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.501018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.521017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.530773] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.548021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.558018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.580025] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.593410] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.605017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.616019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.631028] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.638468] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.647020] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.653035] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.669014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.678317] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.680410] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.682354] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.684270] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.686228] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.689078] iommu: Default domain type: Passthrough [ 0.691568] SCSI subsystem initialized [ 0.693137] ACPI: bus type USB registered [ 0.694096] usbcore: registered new interface driver usbfs [ 0.696081] usbcore: registered new interface driver hub [ 0.697065] usbcore: registered new device driver usb [ 0.699145] pps_core: LinuxPPS API ver. 1 registered [ 0.700010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.702049] PTP clock support registered [ 0.704148] EDAC MC: Ver: 3.0.0 [ 0.706583] PCI: Using ACPI for IRQ routing [ 0.708940] NetLabel: Initializing [ 0.710012] NetLabel: domain hash size = 128 [ 0.712012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.714086] NetLabel: unlabeled traffic allowed by default [ 0.716163] vgaarb: loaded [ 0.718276] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.719014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.729742] clocksource: Switched to clocksource kvm-clock [ 0.844509] VFS: Disk quotas dquot_6.6.0 [ 0.846102] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.848810] *** VALIDATE ramfs *** [ 0.849989] *** VALIDATE hugetlbfs *** [ 0.851202] pnp: PnP ACPI init [ 0.853264] pnp: PnP ACPI: found 6 devices [ 0.868547] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.871865] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.874089] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.876377] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.878948] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.880968] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.883190] NET: Registered protocol family 2 [ 0.885163] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.889495] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.892862] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.896661] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.899914] TCP: Hash tables configured (established 65536 bind 65536) [ 0.902883] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.906087] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.909212] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.912434] NET: Registered protocol family 1 [ 0.915199] RPC: Registered named UNIX socket transport module. [ 0.917795] RPC: Registered udp transport module. [ 0.919578] RPC: Registered tcp transport module. [ 0.920910] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.922571] NET: Registered protocol family 44 [ 0.924137] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.926491] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.928965] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.931833] PCI: CLS 0 bytes, default 64 [ 0.934394] Unpacking initramfs... [ 2.338829] debug: unmapping init [mem 0xffff8ab6fcc54000-0xffff8ab6fffbffff] [ 2.343248] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.345743] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.348660] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.840885] Initialise system trusted keyrings [ 2.842436] Key type blacklist registered [ 2.844219] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.852488] zbud: loaded [ 2.855214] *** VALIDATE nfs *** [ 2.856266] *** VALIDATE nfs4 *** [ 2.857739] pstore: using deflate compression [ 2.864034] Platform Keyring initialized [ 2.984903] NET: Registered protocol family 38 [ 2.986702] Key type asymmetric registered [ 2.988137] Asymmetric key parser 'x509' registered [ 2.990018] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.993172] io scheduler mq-deadline registered [ 2.995071] io scheduler kyber registered [ 2.996645] io scheduler bfq registered [ 2.998423] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.001514] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.004437] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.007329] ACPI: Power Button [PWRF] [ 3.012656] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.019759] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.028698] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.037867] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.053944] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.082271] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.110492] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.114813] Non-volatile memory driver v1.3 [ 3.116224] Linux agpgart interface v0.103 [ 3.153269] virtio_blk virtio1: [vda] 146992 512-byte logical blocks (75.3 MB/71.8 MiB) [ 3.158021] vda: detected capacity change from 0 to 75259904 [ 3.173200] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.176849] vdb: detected capacity change from 0 to 1073741824 [ 3.192158] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.195082] vdc: detected capacity change from 0 to 2621440000 [ 3.213746] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.216934] vdd: detected capacity change from 0 to 2621440000 [ 3.234987] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.237113] vde: detected capacity change from 0 to 4294967296 [ 3.251796] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.253832] vdf: detected capacity change from 0 to 4294967296 [ 3.258766] libphy: Fixed MDIO Bus: probed [ 3.265154] usbcore: registered new interface driver usbserial_generic [ 3.268760] usbserial: USB Serial support registered for generic [ 3.270261] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.273344] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.274570] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.276133] mousedev: PS/2 mouse device common for all mice [ 3.278419] rtc_cmos 00:05: RTC can wake from S4 [ 3.281076] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.281177] rtc_cmos 00:05: registered as rtc0 [ 3.287870] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.287961] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.293943] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.295176] intel_pstate: CPU model not supported [ 3.297299] hid: raw HID events driver (C) Jiri Kosina [ 3.305815] usbcore: registered new interface driver usbhid [ 3.308133] usbhid: USB HID core driver [ 3.309824] drop_monitor: Initializing network drop monitor service [ 3.312246] Initializing XFRM netlink socket [ 3.314369] NET: Registered protocol family 10 [ 3.317532] Segment Routing with IPv6 [ 3.319149] NET: Registered protocol family 17 [ 3.321312] mpls_gso: MPLS GSO support [ 3.327770] RAS: Correctable Errors collector initialized. [ 3.330535] AVX version of gcm_enc/dec engaged. [ 3.332074] AES CTR mode by8 optimization enabled [ 3.412643] sched_clock: Marking stable (3412620054, 0)->(4296288284, -883668230) [ 3.416242] registered taskstats version 1 [ 3.418623] Loading compiled-in X.509 certificates [ 3.420371] zswap: loaded using pool lzo/zbud [ 3.447757] Key type big_key registered [ 3.459718] Key type encrypted registered [ 3.461344] ima: No TPM chip found, activating TPM-bypass! [ 3.463827] ima: Allocated hash algorithm: sha1 [ 3.465848] ima: No architecture policies found [ 3.467594] evm: Initialising EVM extended attributes: [ 3.469149] evm: security.selinux [ 3.470129] evm: security.ima [ 3.470793] evm: security.capability [ 3.471743] evm: HMAC attrs: 0x1 [ 3.474000] rtc_cmos 00:05: setting system clock to 2026-08-27 20:57:47 UTC (1787864267) [ 3.480518] debug: unmapping init [mem 0xffffffffb0603000-0xffffffffb07fffff] [ 3.483128] debug: unmapping init [mem 0xffffffffaf382000-0xffffffffaf658fff] [ 3.489161] Write protecting the kernel read-only data: 28672k [ 3.492101] debug: unmapping init [mem 0xffffffffada03000-0xffffffffadbfffff] [ 3.494765] debug: unmapping init [mem 0xffffffffae314000-0xffffffffae3fffff] [ 3.534119] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.542417] systemd[1]: Detected virtualization kvm. [ 3.544766] systemd[1]: Detected architecture x86-64. [ 3.546332] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.577646] systemd[1]: No hostname configured. [ 3.579524] systemd[1]: Set hostname to . [ 3.582212] random: systemd: uninitialized urandom read (16 bytes read) [ 3.586049] systemd[1]: Initializing machine ID from random generator. [ 3.641166] random: ln: uninitialized urandom read (6 bytes read) [ 3.756915] random: systemd: uninitialized urandom read (16 bytes read) [ 3.759706] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.764912] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.771445] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Paths. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.439488] device-mapper: uevent: version 1.0.3 [ 4.441761] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.201664] virtio_net virtio0 ens2: renamed from eth0 [ 5.222197] random: fast init done [ 5.264545] scsi host0: ata_piix [ 5.278523] scsi host1: ata_piix [ 5.280390] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.283161] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.102142] dracut-initqueue[582]: RTNETLINK answers: File exists [ 9.978563] random: crng init done [ 9.979942] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.521328] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.742856] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.985503] SELinux: Disabled at runtime. [ 12.067859] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.077331] systemd[1]: Detected virtualization kvm. [ 12.079231] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.596165] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.601138] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.605176] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.609525] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.613649] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.621615] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.625347] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted V[ 12.702091] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS olumes. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Slices. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. Starting udev Coldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... Mounting POSIX Message Queue File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.206751] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.525130] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.600077] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.752459] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.768336] EDAC sbridge: Ver: 1.1.2 [ 15.620870] Key type dns_resolver registered [ 15.967471] NFS: Registering the id_resolver key type [ 15.969281] Key type id_resolver registered [ 15.970956] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg110-server login: [ 70.885119] libcfs: loading out-of-tree module taints kernel. [ 71.020208] Key type ._llcrypt registered [ 71.021968] Key type .llcrypt registered [ 71.262839] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 93.607802] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 95.301975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 95.314507] alg: No test for adler32 (adler32-zlib) [ 96.883420] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 97.863192] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 99.663534] Key type lgssc registered [ 102.175928] Lustre: Echo OBD driver; http://www.lustre.org/ [ 122.686362] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 165.027159] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 178.747791] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 178.797181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 180.134653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 180.186643] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 180.291800] Lustre: lustre-MDT0000: new disk, initializing [ 180.425794] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 180.451421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 184.729740] hrtimer: interrupt took 3659319 ns [ 186.150986] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 200.533456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 200.675883] Lustre: 6507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 200.709591] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 200.713155] Lustre: Skipped 1 previous similar message [ 200.796697] Lustre: lustre-MDT0001: new disk, initializing [ 200.907950] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 200.968196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 200.986856] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 206.849707] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.740021] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 222.126753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 222.422301] Lustre: lustre-OST0000: new disk, initializing [ 222.427315] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 222.442626] Lustre: 8458:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 222.583251] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 223.782915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 223.806082] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 223.957913] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 229.052899] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.751449] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 244.890590] Lustre: lustre-OST0001: new disk, initializing [ 244.894189] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 244.905982] Lustre: 9531:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 245.006304] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 251.968172] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 253.004214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 253.015562] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 253.102936] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 264.059753] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 272.063858] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 280.034855] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing check_logdir /tmp/testlogs/ [ 286.901360] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing yml_node [ 291.673333] Lustre: DEBUG MARKER: Client: 2.17.57.106 [ 294.394676] Lustre: DEBUG MARKER: MDS: 2.17.57.106 [ 296.538849] Lustre: DEBUG MARKER: OSS: 2.17.57.106 [ 298.117501] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 17:02:40 EDT 2026 [ 315.086250] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 317.196288] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 334.304644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 334.310759] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 334.339631] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 335.333846] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 335.340717] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 335.357026] Lustre: Skipped 3 previous similar messages [ 340.448148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 340.452836] Lustre: Skipped 3 previous similar messages [ 345.569426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 348.127193] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 348.396460] Lustre: server umount lustre-MDT0000 complete [ 352.912173] LustreError: 6499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864616 with bad export cookie 9908531877997435640 [ 352.923024] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 352.926907] LustreError: 6499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 355.810232] LustreError: 12271:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 355.810488] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 355.828714] LustreError: 12271:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 355.859148] Lustre: Skipped 2 previous similar messages [ 355.868051] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 355.870986] Lustre: Skipped 3 previous similar messages [ 357.858558] LustreError: 6514:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 357.890153] LustreError: 6514:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 359.307950] Lustre: server umount lustre-MDT0001 complete [ 363.314538] Lustre: server umount lustre-OST0000 complete [ 367.089733] Lustre: server umount lustre-OST0001 complete [ 373.896483] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 383.444608] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 422.133543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 422.390819] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 422.459038] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 422.515931] Lustre: lustre-MDT0000: new disk, initializing [ 422.575359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 422.600257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 427.198464] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 438.008129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 438.118596] Lustre: 15085:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 438.165500] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 438.168963] Lustre: Skipped 1 previous similar message [ 438.226184] Lustre: lustre-MDT0001: new disk, initializing [ 438.345490] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 438.426495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 438.438050] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 444.530111] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 455.144173] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 461.128926] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 467.992790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 468.252293] Lustre: lustre-OST0000: new disk, initializing [ 468.257432] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 468.261677] Lustre: 17054:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 468.391523] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 470.086103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 470.100764] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 470.287548] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 474.498947] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 483.628598] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 488.528619] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 488.704426] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 493.304026] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 493.565779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 496.104569] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 496.109821] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 496.119889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 496.119898] Lustre: Skipped 2 previous similar messages [ 496.150806] Lustre: Skipped 2 previous similar messages [ 501.618959] Lustre: server umount lustre-OST0000 complete [ 506.345248] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 506.354683] Lustre: Skipped 1 previous similar message [ 512.578441] Lustre: server umount lustre-MDT0000 complete [ 516.517317] LustreError: 15076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864780 with bad export cookie 9908531877997437173 [ 516.524740] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 516.531611] LustreError: 15076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 516.833856] Lustre: server umount lustre-MDT0001 complete [ 522.372605] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 17:06:24 (1787864784) [ 529.547523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 529.987285] LustreError: 19317:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 530.087698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 535.274262] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 535.523953] LustreError: 19318:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 535.563111] LustreError: 19318:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 540.644355] LustreError: 19317:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.502979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 549.720566] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 557.521972] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 564.122934] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 571.221056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 578.455514] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 586.010936] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 593.270888] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 601.058543] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 601.066522] LustreError: Skipped 1 previous similar message [ 601.072900] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 601.088598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 601.098409] Lustre: Skipped 7 previous similar messages [ 613.346062] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 613.738757] Lustre: server umount lustre-MDT0000 complete [ 616.936214] LustreError: 19318:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 616.960181] LustreError: 19318:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 617.382382] LustreError: 19299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864881 with bad export cookie 9908531877997438055 [ 617.401753] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 617.691698] Lustre: server umount lustre-MDT0001 complete [ 632.087598] Lustre: server umount lustre-OST0000 complete [ 659.963639] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 672.647838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 673.081817] LustreError: 23702:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 673.119371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 673.129119] Lustre: Skipped 2 previous similar messages [ 678.202184] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 689.804062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 696.532551] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 700.682526] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 708.348891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 715.126844] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 723.178861] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 723.469228] Lustre: lustre-OST0001: new disk, initializing [ 723.472616] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 723.478659] Lustre: 26032:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 724.989519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 725.003203] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 725.136853] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 729.621980] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 739.122631] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 742.862175] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 755.686792] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 755.688103] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 755.706528] Lustre: Skipped 4 previous similar messages [ 755.717377] Lustre: Skipped 10 previous similar messages [ 760.964688] Lustre: server umount lustre-MDT0000 complete [ 764.700885] LustreError: 23682:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865028 with bad export cookie 9908531877997438902 [ 764.710379] LustreError: 23682:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 764.714372] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 765.002605] Lustre: server umount lustre-MDT0001 complete [ 778.208565] Lustre: server umount lustre-OST0000 complete [ 781.282085] Lustre: 3642:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787865029/real 1787865029] req@ffff8ab759a05a40 x1874711660449664/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787865045 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 782.231253] Lustre: server umount lustre-OST0001 complete [ 800.844308] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 811.558174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 811.984735] LustreError: 29050:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 812.007617] LustreError: 29050:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 812.116214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 812.131442] Lustre: Skipped 3 previous similar messages [ 817.069389] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 827.855801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 833.388587] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 836.928348] Lustre: 30197:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 844.435601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 854.023255] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 863.509380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 871.002973] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 881.264693] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 886.036469] Lustre: 32077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 894.439664] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 894.443196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 894.466796] Lustre: Skipped 4 previous similar messages [ 894.487199] Lustre: Skipped 7 previous similar messages [ 899.205778] Lustre: server umount lustre-MDT0000 complete [ 899.574147] LustreError: 29047:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 899.608680] LustreError: 29047:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 12 previous similar messages [ 903.824423] LustreError: 29032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865167 with bad export cookie 9908531877997440197 [ 903.832456] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 903.837433] LustreError: 29032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 904.225538] Lustre: server umount lustre-MDT0001 complete [ 919.325560] Lustre: server umount lustre-OST0000 complete [ 920.095233] Lustre: 3640:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787865168/real 1787865168] req@ffff8ab74430f0c0 x1874711660504192/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787865184 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 924.469764] Lustre: server umount lustre-OST0001 complete [ 935.071880] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 17:13:17 (1787865197) [ 971.718988] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 983.920849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 990.345100] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 999.404902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1004.686604] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1007.457710] Lustre: 36031:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1014.039517] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1022.251040] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1029.993164] LustreError: 36387:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1030.021615] LustreError: 36387:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 1031.071766] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1037.517800] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1045.379537] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1049.259786] Lustre: 37910:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1056.225908] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1056.237691] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1056.259914] Lustre: Skipped 2 previous similar messages [ 1056.263410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1061.118318] Lustre: server umount lustre-MDT0000 complete [ 1064.907263] LustreError: 34863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865328 with bad export cookie 9908531877997441303 [ 1064.914567] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1064.918543] LustreError: 34863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1065.322713] Lustre: server umount lustre-MDT0001 complete [ 1079.758323] Lustre: server umount lustre-OST0000 complete [ 1083.872084] Lustre: 3641:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787865331/real 1787865331] req@ffff8ab647ee1a40 x1874711660553088/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787865347 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1084.326104] Lustre: server umount lustre-OST0001 complete [ 1093.730521] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 17:15:56 (1787865356) [ 1111.555574] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 1122.354186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.912636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1122.918858] Lustre: Skipped 7 previous similar messages [ 1127.800485] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1136.924472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1142.966326] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1146.104306] Lustre: 41199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1153.467803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1160.699711] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1170.529971] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1177.805720] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1185.682508] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1189.468524] Lustre: 43076:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1190.572696] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1190.583372] Lustre: Skipped 1 previous similar message [ 1193.953835] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1193.963580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1193.975393] Lustre: Skipped 4 previous similar messages [ 1199.895761] Lustre: server umount lustre-MDT0000 complete [ 1203.740977] LustreError: 40032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865467 with bad export cookie 9908531877997442402 [ 1203.747986] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1203.757146] LustreError: 40032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1210.281330] Lustre: server umount lustre-MDT0001 complete [ 1219.045661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1228.270172] LustreError: 3639:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8ab746a1ed00 x1874711660604928/t0(0) o250->MGC192.168.201.110@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1233.483927] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1238.315221] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1243.163904] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1248.970911] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1249.310926] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1249.328527] Lustre: Skipped 1 previous similar message [ 1257.664528] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1259.595540] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1265.514789] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1267.127529] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1274.339570] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1278.808773] Lustre: server umount lustre-MDT0000 complete [ 1282.859783] LustreError: 40033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865546 with bad export cookie 9908531877997443508 [ 1282.869113] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1282.872055] LustreError: 40033:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1283.290430] Lustre: server umount lustre-MDT0001 complete [ 1297.792784] Lustre: server umount lustre-OST0000 complete [ 1311.966991] Lustre: server umount lustre-OST0001 complete [ 1321.048488] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 17:19:43 (1787865583) [ 1341.509490] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 1355.060741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1355.525471] LustreError: 47690:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1355.540530] LustreError: 47690:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 34 previous similar messages [ 1361.531134] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1371.685136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1376.413551] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1379.494175] Lustre: 48844:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1387.622578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1394.500988] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1404.769684] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1411.875167] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1419.808685] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1424.700275] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1438.688493] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1438.710964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1438.723415] Lustre: Skipped 19 previous similar messages [ 1444.685304] Lustre: server umount lustre-MDT0000 complete [ 1450.037405] LustreError: 47671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865714 with bad export cookie 9908531877997444334 [ 1450.062726] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1450.493118] Lustre: server umount lustre-MDT0001 complete [ 1465.953822] Lustre: server umount lustre-OST0000 complete [ 1467.874318] Lustre: 3642:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787865715/real 1787865715] req@ffff8ab77f1a4f00 x1874711660699904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787865731 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1467.896258] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1467.903720] Lustre: Skipped 13 previous similar messages [ 1469.773643] Lustre: server umount lustre-OST0001 complete [ 1479.393445] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 17:22:21 (1787865741) [ 1482.443414] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1484.406140] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 17:22:27 (1787865747) [ 1495.160417] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 1502.858594] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 1524.997636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1525.240887] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1525.293623] Lustre: lustre-MDT0000: new disk, initializing [ 1525.361671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1529.816946] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1538.069345] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1544.474265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1544.803786] Lustre: lustre-OST0000: new disk, initializing [ 1544.807333] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1544.811867] Lustre: Skipped 1 previous similar message [ 1544.815544] Lustre: 54918:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1546.308737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1546.322521] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1546.374357] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1551.963305] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1562.104375] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1565.097026] Lustre: DEBUG MARKER: create test files [ 1602.529873] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1608.265565] Lustre: server umount lustre-OST0000 complete [ 1612.516584] Lustre: server umount lustre-MDT0000 complete [ 1623.915689] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 1626.607933] Key type lgssc unregistered [ 1626.833355] LNet: 56559:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1626.841495] LNetError: 56559:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1626.856479] LNet: Removed LNI 192.168.201.110@tcp [ 1627.893281] Key type .llcrypt unregistered [ 1627.896309] Key type ._llcrypt unregistered [ 1652.540226] Key type ._llcrypt registered [ 1652.543371] Key type .llcrypt registered [ 1652.662501] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 1653.651521] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1653.738173] alg: No test for adler32 (adler32-zlib) [ 1654.795782] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 1655.009665] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 1656.687173] Key type lgssc registered [ 1657.806305] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1666.422137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1666.466435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1667.919642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1672.836839] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1680.100906] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1687.899228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1688.413032] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1692.550767] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1692.613516] Lustre: 59274:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1692.619683] Lustre: 59274:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1692.623912] Lustre: 59274:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1692.627743] Lustre: 59274:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1692.634617] Lustre: 59274:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1692.638517] Lustre: 59274:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1696.536902] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1704.254704] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1709.452831] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1714.340098] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1723.368417] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1723.374386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1723.387135] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1723.854572] Lustre: server umount lustre-OST0000 complete [ 1727.592986] Lustre: server umount lustre-MDT0000 complete [ 1738.549585] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 1741.307415] Key type lgssc unregistered [ 1741.583400] LNet: 60885:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1741.598860] LNetError: 60885:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1741.630651] LNet: Removed LNI 192.168.201.110@tcp [ 1742.709433] Key type .llcrypt unregistered [ 1742.714765] Key type ._llcrypt unregistered [ 1773.790756] Key type ._llcrypt registered [ 1773.797026] Key type .llcrypt registered [ 1773.920152] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 1775.448586] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1775.474753] alg: No test for adler32 (adler32-zlib) [ 1776.462868] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 1776.712285] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 1778.412737] Key type lgssc registered [ 1779.788593] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1787.799943] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1787.830920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1789.491351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1794.198885] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1802.790637] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1812.620775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1813.402430] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1814.460120] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1814.646739] Lustre: 63722:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1814.660362] Lustre: 63722:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1814.669227] Lustre: 63722:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1814.677450] Lustre: 63722:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1814.686937] Lustre: 63722:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1814.690550] Lustre: 63722:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1821.532173] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1831.040734] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1836.852436] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1875.946094] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1875.961225] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1875.961663] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1880.218955] Lustre: server umount lustre-OST0000 complete [ 1884.603420] Lustre: server umount lustre-MDT0000 complete [ 1895.328573] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 1898.033572] Key type lgssc unregistered [ 1898.319709] LNet: 65310:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1898.322982] LNetError: 65310:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1899.367298] LNet: Removed LNI 192.168.201.110@tcp [ 1900.269514] Key type .llcrypt unregistered [ 1900.272365] Key type ._llcrypt unregistered [ 1917.937593] Key type ._llcrypt registered [ 1917.946525] Key type .llcrypt registered [ 1918.076946] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 1931.188576] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 1932.246235] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1932.530864] alg: No test for adler32 (adler32-zlib) [ 1933.746331] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 1934.111032] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 1935.767665] Key type lgssc registered [ 1937.488025] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1976.750618] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1976.797711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1978.288585] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1978.372487] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1978.453626] Lustre: lustre-MDT0000: new disk, initializing [ 1978.503950] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1978.532952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1984.010965] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 1996.614264] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1996.833166] Lustre: 68460:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1996.958714] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1996.965574] Lustre: Skipped 1 previous similar message [ 1997.116862] Lustre: lustre-MDT0001: new disk, initializing [ 1997.411963] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1997.464179] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1997.496244] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2002.172081] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2013.819749] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2020.435188] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2026.916450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2027.158177] Lustre: lustre-OST0000: new disk, initializing [ 2027.165857] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2027.177347] Lustre: 70428:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2027.313746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2033.655920] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2033.761939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2033.779082] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2033.916381] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2043.364612] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2047.538934] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2047.761330] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2051.647431] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2051.820625] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2054.628276] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2054.630913] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2054.637622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2054.647787] Lustre: Skipped 1 previous similar message [ 2054.665441] LustreError: Skipped 1 previous similar message [ 2059.629149] Lustre: server umount lustre-OST0000 complete [ 2063.840936] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2063.853733] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2063.874280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2063.876973] Lustre: Skipped 1 previous similar message [ 2064.873858] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2064.905981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2064.920192] Lustre: Skipped 1 previous similar message [ 2069.645664] Lustre: server umount lustre-MDT0000 complete [ 2073.146176] LustreError: 68451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787866337 with bad export cookie 118706730969000303 [ 2073.150449] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2073.157665] LustreError: 68451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2073.503427] Lustre: server umount lustre-MDT0001 complete [ 2081.650684] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 17:32:24 (1787866344) [ 2091.778760] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 2131.801245] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 2140.358074] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 2173.200987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2173.383120] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2173.398417] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2173.445306] Lustre: lustre-MDT0000: new disk, initializing [ 2173.497172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2173.511963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2177.871272] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2189.197150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2189.260581] Lustre: 76445:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2189.295122] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2189.297778] Lustre: Skipped 1 previous similar message [ 2189.348472] Lustre: lustre-MDT0001: new disk, initializing [ 2189.410870] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2189.431984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2189.444852] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2194.093402] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2205.666936] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2210.559057] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2216.671059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2216.948703] Lustre: lustre-OST0000: new disk, initializing [ 2216.958885] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2216.970931] Lustre: 78413:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2217.040264] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2222.289056] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2225.770623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2225.803216] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2225.920385] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2231.015487] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2235.693837] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2235.866793] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2240.331476] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2240.504403] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2246.624140] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2246.630501] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2246.645050] Lustre: Skipped 1 previous similar message [ 2246.646645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2246.661058] Lustre: Skipped 1 previous similar message [ 2248.357435] Lustre: server umount lustre-OST0000 complete [ 2255.848898] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2255.865228] LustreError: Skipped 1 previous similar message [ 2255.870169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2255.888591] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2258.848946] Lustre: server umount lustre-MDT0000 complete [ 2261.992390] LustreError: 76456:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2262.015481] LustreError: 76456:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 2262.579935] LustreError: 76437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787866526 with bad export cookie 118706730969001178 [ 2262.582724] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2262.596346] LustreError: 76437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2263.021524] Lustre: server umount lustre-MDT0001 complete [ 2272.584404] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 17:35:35 (1787866535) [ 2279.188768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2279.612120] LustreError: 80827:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2279.694939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2284.214990] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2285.031699] LustreError: 80828:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2289.121956] LustreError: 80827:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2292.665384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.038119] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2304.287126] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2309.476421] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2315.126383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2315.409172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2315.417376] Lustre: Skipped 1 previous similar message [ 2321.469447] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2327.857453] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2330.800535] Lustre: 83498:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 5fdc56cd-bd9a-4bf0-8d09-9a79ec581cc3 at adminstrative request [ 2335.215917] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2335.231389] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2345.445809] Lustre: MGS: Client 5fdc56cd-bd9a-4bf0-8d09-9a79ec581cc3 (at 0@lo) reconnecting [ 2345.458945] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1a5bb2493daf053 to 0x1a5bb2493daf2f3 [ 2345.480252] Lustre: MGC192.168.201.110@tcp: Connection restored to 0@lo (at 0@lo) [ 2367.132851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2367.339390] Lustre: lustre-OST0001: new disk, initializing [ 2367.342678] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2367.355173] Lustre: 83890:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2373.104798] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2375.270203] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2375.276291] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2375.364161] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2381.120067] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2385.904067] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2385.905678] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2385.915365] Lustre: Skipped 2 previous similar messages [ 2385.920207] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2385.926438] Lustre: Skipped 3 previous similar messages [ 2389.367872] Lustre: server umount lustre-OST0001 complete [ 2391.012756] LustreError: 82647:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2394.596669] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2394.604761] LustreError: Skipped 1 previous similar message [ 2398.782745] Lustre: server umount lustre-OST0000 complete [ 2405.857540] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2405.867730] Lustre: Skipped 2 previous similar messages [ 2405.872794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2405.877727] Lustre: Skipped 3 previous similar messages [ 2408.571954] Lustre: server umount lustre-MDT0000 complete [ 2411.495162] LustreError: 81554:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2411.518743] LustreError: 81554:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 2412.420720] LustreError: 83889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787866676 with bad export cookie 118706730969002739 [ 2412.423542] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2412.433321] LustreError: 83889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2412.659838] Lustre: server umount lustre-MDT0001 complete [ 2421.425812] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 17:38:04 (1787866684) [ 2422.849104] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2424.657299] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 17:38:07 (1787866687) [ 2426.398487] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2427.902848] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 17:38:10 (1787866690) [ 2429.457231] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2431.222241] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 17:38:13 (1787866693) [ 2432.607379] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2434.194842] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 17:38:17 (1787866697) [ 2435.945178] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2441.805713] loop: module loaded [ 2441.832510] loop0: detected capacity change from 0 to 4294967296 [ 2441.926244] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2443.803668] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2447.498824] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2452.289710] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2486.388200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2486.425863] LustreError: 88008:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2486.432538] LustreError: 88008:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2486.436304] LustreError: 88008:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2486.440339] LustreError: 88008:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2506.064796] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 2514.320716] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 2546.929674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2547.148451] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2547.171518] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2547.226286] Lustre: lustre-MDT0000: new disk, initializing [ 2547.280819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2547.283634] Lustre: Skipped 1 previous similar message [ 2547.295433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2551.362673] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2561.864920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2561.980297] Lustre: 90874:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2562.046224] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 2562.055305] Lustre: Skipped 2 previous similar messages [ 2562.352412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2565.839885] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2575.799149] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2582.292266] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2589.422578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2589.674756] Lustre: 92844:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2596.318160] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2599.965488] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2600.069044] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2604.805926] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2610.324848] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2610.501783] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2614.735894] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2614.989838] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2620.384960] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2620.393645] LustreError: Skipped 1 previous similar message [ 2620.403629] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2620.423182] Lustre: Skipped 1 previous similar message [ 2631.135151] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2631.139800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2631.160436] Lustre: Skipped 4 previous similar messages [ 2631.278696] Lustre: server umount lustre-OST0000 complete [ 2640.982452] Lustre: server umount lustre-MDT0000 complete [ 2641.376784] LustreError: 90887:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2641.391137] LustreError: 90887:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2644.763976] LustreError: 90867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787866908 with bad export cookie 118706730969003355 [ 2644.768561] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2644.774692] LustreError: 90867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2645.238873] Lustre: server umount lustre-MDT0001 complete [ 2653.462397] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 17:41:56 (1787866916) [ 2660.215836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2665.287450] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2673.170830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2678.072865] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2686.674251] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2691.582238] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2698.818850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2699.211971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2699.218677] Lustre: Skipped 4 previous similar messages [ 2706.567618] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2715.701504] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2720.650480] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2720.947411] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2725.652382] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2725.868572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2732.203962] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2732.417164] Lustre: lustre-OST0001: new disk, initializing [ 2732.420466] Lustre: Skipped 2 previous similar messages [ 2732.424583] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2732.428260] Lustre: Skipped 3 previous similar messages [ 2732.432616] Lustre: 98623:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2739.844650] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2741.924599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2741.950888] Lustre: Skipped 2 previous similar messages [ 2741.971647] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2742.129928] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2751.141813] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2756.264467] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2756.509560] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2761.141792] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2761.348507] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2780.048628] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2801.350184] Lustre: Failing over lustre-MDT0000 [ 2801.632712] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2801.638259] LustreError: Skipped 1 previous similar message [ 2801.641211] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2801.652139] Lustre: Skipped 3 previous similar messages [ 2801.658284] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2801.663198] Lustre: Skipped 3 previous similar messages [ 2801.754202] Lustre: server umount lustre-MDT0000 complete [ 2803.691149] LustreError: 98838:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2803.705899] LustreError: 98838:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 2819.043667] Lustre: 66682:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787867067/real 1787867067] req@00000000b6d5ea71 x1874713586703744/t0(0) o400->MGC192.168.201.110@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787867083 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2819.086977] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2825.262335] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2825.273992] LDISKFS-fs (dm-0): recovery complete [ 2825.287854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2829.281512] LustreError: 66678:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000042807558 x1874713586711936/t0(0) o250->MGC192.168.201.110@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2829.690290] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2830.183095] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2830.183943] Lustre: 95253:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2834.331845] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 2834.917390] Lustre: 95257:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2834.930518] Lustre: 95257:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 1 previous similar message [ 2834.941393] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2835.084230] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2835.408068] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2835.770870] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2835.779483] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 2836.103781] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2836.120279] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 2836.132737] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2836.153789] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 7 previous similar messages [ 2836.800133] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2836.813263] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 2837.136829] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2837.152106] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 2838.433612] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2838.451509] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 6 previous similar messages [ 2839.095574] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2839.118329] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 2839.439151] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2839.451892] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 2842.647967] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2842.662751] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 12 previous similar messages [ 2843.320663] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2843.341205] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 2843.667555] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2843.672949] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 2850.669261] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2850.693162] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 24 previous similar messages [ 2851.340724] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2851.360217] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 24 previous similar messages [ 2851.687123] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2851.709446] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 24 previous similar messages [ 2866.672728] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2866.686726] Lustre: 101477:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 49 previous similar messages [ 2867.649949] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2867.670161] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 2868.007184] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2868.012873] LustreError: 101477:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 2890.311212] Lustre: 101477:0:(ldlm_lib.c:2457:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2890.320297] Lustre: 101477:0:(ldlm_lib.c:2467:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2890.331568] Lustre: 101477:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 3304ed74-1758-4a0b-b754-48182e782634@192.168.201.10@tcp [ 2890.349341] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2890.353835] Lustre: 101477:0:(ldlm_lib.c:1977:abort_req_replay_queue()) @@@ aborted: req@00000000a8d101ff x1874713571821440/t0(8589934769) o101->3304ed74-1758-4a0b-b754-48182e782634@192.168.201.10@tcp:740/0 lens 592/0 e 0 to 0 dl 1787867165 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2890.378887] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2890.389995] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2890.394342] Lustre: lustre-MDT0000: Denying connection for new client 3304ed74-1758-4a0b-b754-48182e782634 (at 192.168.201.10@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2890.432014] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2890.437426] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2890.442199] Lustre: Skipped 2 previous similar messages [ 2890.494308] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2890.505755] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2906.592861] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2912.203441] Lustre: server umount lustre-OST0000 complete [ 2916.832468] LustreError: 100026:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2916.847269] LustreError: 100026:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 31 previous similar messages [ 2922.483935] Lustre: server umount lustre-OST0001 complete [ 2932.917413] Lustre: server umount lustre-MDT0000 complete [ 2937.073944] LustreError: 100726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787867201 with bad export cookie 118706730969054567 [ 2937.083631] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2937.088717] LustreError: 100726:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2937.652821] Lustre: server umount lustre-MDT0001 complete [ 2945.549562] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 17:46:48 (1787867208) [ 2952.989966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2952.995596] Lustre: *** cfs_fail_loc=197, val=0*** [ 2968.607635] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2973.727470] LustreError: 103451:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.110@tcp: failed processing log, type 4: rc = -110 [ 2978.935595] Lustre: *** cfs_fail_loc=197, val=0*** [ 2978.939466] Lustre: Skipped 65 previous similar messages [ 2999.521549] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2999.530050] Lustre: Skipped 2 previous similar messages [ 3005.041500] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3010.567553] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3012.258572] Lustre: server umount lustre-OST0000 complete [ 3020.804232] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 17:48:03 (1787867283) [ 3041.511337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3046.441780] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3047.392562] LustreError: 105072:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3047.402498] LustreError: 105072:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 3055.468391] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3059.345215] LustreError: 105102:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3059.351636] LustreError: 105102:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3059.360681] LustreError: 105102:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3059.645627] Lustre: server umount lustre-MDT0000 complete [ 3097.944721] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 3108.969786] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 3139.589832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3139.908472] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3139.959404] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3140.069277] Lustre: lustre-MDT0000: new disk, initializing [ 3140.169363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3144.358960] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3155.772178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3155.921046] Lustre: 109276:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3156.148430] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3161.264423] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3170.842908] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3175.898969] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3183.268893] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3183.553626] Lustre: 111248:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3189.872554] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3193.113460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3200.982712] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3207.264092] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3207.518811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3213.019419] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3213.273881] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3218.921532] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3218.926582] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3218.942919] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3218.942927] Lustre: Skipped 13 previous similar messages [ 3218.943278] Lustre: Skipped 10 previous similar messages [ 3218.951284] LustreError: Skipped 3 previous similar messages [ 3222.060878] Lustre: server umount lustre-OST0000 complete [ 3232.641878] Lustre: server umount lustre-MDT0000 complete [ 3236.589663] LustreError: 109270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787867500 with bad export cookie 118706730969065144 [ 3236.613479] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3237.044900] Lustre: server umount lustre-MDT0001 complete [ 3246.419751] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:51:49 (1787867509) [ 3256.156423] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 3260.073319] Key type lgssc unregistered [ 3260.468120] LNet: 113737:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3260.477094] LNetError: 113737:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3260.503146] LNet: Removed LNI 192.168.201.110@tcp [ 3261.367081] Key type .llcrypt unregistered [ 3261.369929] Key type ._llcrypt unregistered [ 3279.834456] Key type ._llcrypt registered [ 3279.837049] Key type .llcrypt registered [ 3279.915884] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 3294.008382] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 3295.150612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3295.335357] alg: No test for adler32 (adler32-zlib) [ 3296.422435] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 3296.699577] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 3298.343420] Key type lgssc registered [ 3299.290393] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3336.551159] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3336.568995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3337.831692] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3337.869969] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3337.958184] Lustre: lustre-MDT0000: new disk, initializing [ 3338.045082] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3338.061677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3342.235941] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3352.678453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3353.049339] Lustre: lustre-OST0000: new disk, initializing [ 3353.062808] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3353.073534] Lustre: Skipped 1 previous similar message [ 3353.079572] Lustre: 117911:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3353.140260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3353.494704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3353.499462] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3353.527043] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3359.057952] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3369.596698] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3369.777478] Lustre: lustre-OST0001: new disk, initializing [ 3369.781618] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3369.794508] Lustre: 118773:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3369.860735] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3375.990340] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3378.302981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3378.313582] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3378.332755] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3388.386110] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3388.389064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3388.407723] Lustre: Skipped 1 previous similar message [ 3388.432502] Lustre: Skipped 1 previous similar message [ 3390.585636] Lustre: server umount lustre-MDT0000 complete [ 3396.900549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3401.877854] LustreError: 117096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787867665 with bad export cookie 5081788204404032519 [ 3401.882165] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3401.897978] LustreError: 117096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3402.000618] Lustre: server umount lustre-OST0000 complete [ 3407.089561] Lustre: server umount lustre-OST0001 complete [ 3417.074052] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 17:54:39 (1787867679) [ 3439.807811] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 17:55:02 (1787867702) [ 3451.680409] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 3474.613871] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:55:37 (1787867737) [ 3511.950660] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 3521.062129] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 3551.088978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3551.297642] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3551.327985] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3551.418933] Lustre: lustre-MDT0000: new disk, initializing [ 3551.489849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3551.510614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3555.797752] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3564.866883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3564.975297] Lustre: 126524:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3565.030662] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3565.032975] Lustre: Skipped 1 previous similar message [ 3565.080349] Lustre: lustre-MDT0001: new disk, initializing [ 3565.155825] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3565.203489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3565.212664] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3568.930029] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3577.856644] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3583.269873] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3589.430597] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3589.677940] Lustre: lustre-OST0000: new disk, initializing [ 3589.681898] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3589.685782] Lustre: 128490:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3589.755916] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3595.596559] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3597.406628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3597.419872] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3597.500421] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3604.335609] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3610.344609] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3610.590653] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3616.088221] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3616.266082] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3623.392387] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3623.406419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3623.419940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3623.429463] Lustre: Skipped 1 previous similar message [ 3623.447667] Lustre: Skipped 1 previous similar message [ 3624.629331] Lustre: server umount lustre-OST0000 complete [ 3631.584529] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3631.596822] LustreError: Skipped 1 previous similar message [ 3631.599869] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3631.618045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3633.638378] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3633.639772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3633.660777] Lustre: Skipped 1 previous similar message [ 3635.387172] Lustre: server umount lustre-MDT0000 complete [ 3638.917650] LustreError: 126515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787867902 with bad export cookie 5081788204404033744 [ 3638.921767] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3638.923019] LustreError: 126515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3639.303257] Lustre: server umount lustre-MDT0001 complete [ 3647.484904] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:58:30 (1787867910) [ 3653.295108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3653.633864] LustreError: 130901:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3653.655953] LustreError: 130901:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 3653.744603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3657.758868] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3658.723293] LustreError: 130902:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3663.841791] LustreError: 130901:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3666.061977] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3671.614620] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3679.835852] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3685.430908] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3692.060065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3692.292871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3692.297246] Lustre: Skipped 1 previous similar message [ 3697.926186] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3704.840915] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3717.064232] Lustre: *** cfs_fail_loc=159, val=0*** [ 3733.377771] Lustre: lustre-MDT0000: Client ca67963a-58ed-43c8-be81-81951cfa3f04 (at 192.168.201.10@tcp) reconnecting [ 3734.634713] Lustre: *** cfs_fail_loc=159, val=0*** [ 3734.636071] Lustre: Skipped 5 previous similar messages [ 3750.296515] Lustre: lustre-MDT0000: Client ca67963a-58ed-43c8-be81-81951cfa3f04 (at 192.168.201.10@tcp) reconnecting [ 3768.289508] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3768.300340] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3768.310662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3768.323547] Lustre: Skipped 1 previous similar message [ 3768.335712] Lustre: Skipped 1 previous similar message [ 3773.820726] Lustre: server umount lustre-OST0000 complete [ 3778.528609] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3778.551863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3778.563220] Lustre: Skipped 3 previous similar messages [ 3791.327782] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3791.613814] Lustre: server umount lustre-MDT0000 complete [ 3793.888106] LustreError: 131629:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3793.916585] LustreError: 131629:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 3795.551531] LustreError: 133520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868059 with bad export cookie 5081788204404034626 [ 3795.557558] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3795.561452] LustreError: 133520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3795.939549] Lustre: server umount lustre-MDT0001 complete [ 3812.728773] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 3816.941723] Key type lgssc unregistered [ 3817.378550] LNet: 134714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3817.396886] LNetError: 134714:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3817.425498] LNet: Removed LNI 192.168.201.110@tcp [ 3818.778094] Key type .llcrypt unregistered [ 3818.781850] Key type ._llcrypt unregistered [ 3843.852591] Key type ._llcrypt registered [ 3843.855947] Key type .llcrypt registered [ 3843.976422] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 3845.030587] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3845.227634] alg: No test for adler32 (adler32-zlib) [ 3846.351817] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 3846.606936] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 3848.271198] Key type lgssc registered [ 3849.311434] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3856.595268] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3856.614164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3858.097474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3862.636251] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3868.647813] LustreError: 136131:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3868.668392] LustreError: 136131:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 3870.760503] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3871.176700] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3875.869712] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3882.425946] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3887.450796] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3893.856390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3894.190741] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3900.553510] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 3903.417513] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3907.331956] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3915.503829] Lustre: *** cfs_fail_loc=159, val=0*** [ 3931.024728] Lustre: lustre-MDT0000: Client d0eabe30-0ead-42d9-bfcf-da6ff691c607 (at 192.168.201.10@tcp) reconnecting [ 3933.917696] Lustre: *** cfs_fail_loc=159, val=0*** [ 3933.932840] Lustre: Skipped 1 previous similar message [ 3950.468625] Lustre: lustre-MDT0001: Client d0eabe30-0ead-42d9-bfcf-da6ff691c607 (at 192.168.201.10@tcp) reconnecting [ 3952.525343] Lustre: *** cfs_fail_loc=159, val=0*** [ 3952.533795] Lustre: Skipped 3 previous similar messages [ 3967.907954] Lustre: lustre-MDT0001: Client d0eabe30-0ead-42d9-bfcf-da6ff691c607 (at 192.168.201.10@tcp) reconnecting [ 3972.640746] Lustre: 139169:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3975.321547] Lustre: *** cfs_fail_loc=159, val=0*** [ 3975.335268] Lustre: Skipped 4 previous similar messages [ 3991.200818] Lustre: lustre-MDT0000: Client 28d9464d-1c7b-4a5f-a132-986ca02d988a (at 192.168.201.10@tcp) reconnecting [ 3993.957893] Lustre: *** cfs_fail_loc=159, val=0*** [ 3993.966829] Lustre: Skipped 13 previous similar messages [ 4010.391348] Lustre: lustre-MDT0000: Client 28d9464d-1c7b-4a5f-a132-986ca02d988a (at 192.168.201.10@tcp) reconnecting [ 4012.077646] Lustre: 139456:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4018.153369] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4018.166396] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4018.203678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4021.218877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4021.221708] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4021.253579] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4021.277298] LustreError: Skipped 2 previous similar messages [ 4024.332920] Lustre: server umount lustre-OST0000 complete [ 4029.924701] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4029.944454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4029.958949] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4029.967836] Lustre: Skipped 1 previous similar message [ 4034.288854] Lustre: server umount lustre-MDT0000 complete [ 4036.584603] LustreError: 139228:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4036.611135] LustreError: 139228:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4038.040789] LustreError: 136111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868302 with bad export cookie 6931343125008352449 [ 4038.046923] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4038.059645] LustreError: 136111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4038.248623] Lustre: server umount lustre-MDT0001 complete [ 4052.123613] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 4055.546891] Key type lgssc unregistered [ 4055.853631] LNet: 140511:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4055.858821] LNetError: 140511:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4055.873581] LNet: Removed LNI 192.168.201.110@tcp [ 4056.891939] Key type .llcrypt unregistered [ 4056.893811] Key type ._llcrypt unregistered [ 4088.481512] Key type ._llcrypt registered [ 4088.487416] Key type .llcrypt registered [ 4088.657274] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 4090.100914] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4090.389703] alg: No test for adler32 (adler32-zlib) [ 4091.427996] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 4091.654558] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 4093.319179] Key type lgssc registered [ 4094.739433] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4103.543584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4103.560766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4105.149516] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4110.426433] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4115.432903] LustreError: 141927:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4115.456886] LustreError: 141927:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4120.546503] LustreError: 141928:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4120.886790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4121.281406] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4126.609626] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4134.964447] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4140.646444] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4146.856791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4147.154516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4150.285133] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4153.916302] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4161.227259] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4166.696983] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4170.759793] Lustre: 144513:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4173.873475] Lustre: 144562:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4175.319294] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4176.262350] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4180.260549] Lustre: 144708:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4181.206324] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4182.300747] Lustre: 144816:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4187.104977] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4187.106855] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4187.140830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4192.298903] LustreError: 143749:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4192.440702] Lustre: server umount lustre-OST0000 complete [ 4197.347401] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4197.350877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4197.367569] Lustre: Skipped 2 previous similar messages [ 4197.385853] Lustre: Skipped 1 previous similar message [ 4202.457660] Lustre: server umount lustre-MDT0000 complete [ 4202.477231] LustreError: 143094:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4202.508879] LustreError: 143094:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4206.597944] LustreError: 142663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868470 with bad export cookie 4730822641939659232 [ 4206.606827] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4206.614142] LustreError: 142663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4206.906480] Lustre: server umount lustre-MDT0001 complete [ 4219.680156] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 4222.421828] Key type lgssc unregistered [ 4222.671634] LNet: 145875:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4222.677141] LNetError: 145875:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4223.721823] LNet: Removed LNI 192.168.201.110@tcp [ 4224.404762] Key type .llcrypt unregistered [ 4224.406486] Key type ._llcrypt unregistered [ 4249.060859] Key type ._llcrypt registered [ 4249.063403] Key type .llcrypt registered [ 4249.207384] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 4251.382925] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4251.457489] alg: No test for adler32 (adler32-zlib) [ 4252.558387] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 4252.859336] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 4254.649913] Key type lgssc registered [ 4255.996719] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4264.994526] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4265.010256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4266.323800] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4271.079508] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4276.705377] LustreError: 147293:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4276.725306] LustreError: 147293:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4280.150239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4280.536741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4285.006212] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4291.902628] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4297.914640] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4305.806488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4306.099885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4311.248624] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4311.824794] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4319.332926] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4325.616221] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4329.288179] Lustre: *** cfs_fail_loc=159, val=0*** [ 4345.757429] Lustre: lustre-MDT0000: Client a46eb7c8-c825-4d09-b585-5884bec80574 (at 192.168.201.10@tcp) reconnecting [ 4351.458799] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4351.477932] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4351.484672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4356.069681] Lustre: server umount lustre-OST0000 complete [ 4361.696413] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4361.702323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4361.715787] Lustre: Skipped 2 previous similar messages [ 4361.735343] Lustre: Skipped 2 previous similar messages [ 4366.825900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4366.849683] Lustre: Skipped 1 previous similar message [ 4367.717823] Lustre: server umount lustre-MDT0000 complete [ 4371.940079] LustreError: 149901:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4371.991816] LustreError: 149901:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4373.713496] LustreError: 149143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868637 with bad export cookie 7270446412959207810 [ 4373.717434] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4373.731493] LustreError: 149143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4374.422810] Lustre: server umount lustre-MDT0001 complete [ 4387.108110] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 4390.023778] Key type lgssc unregistered [ 4390.474984] LNet: 150959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4390.486432] LNetError: 150959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4390.512337] LNet: Removed LNI 192.168.201.110@tcp [ 4391.543138] Key type .llcrypt unregistered [ 4391.545545] Key type ._llcrypt unregistered [ 4415.864217] Key type ._llcrypt registered [ 4415.869748] Key type .llcrypt registered [ 4415.987257] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 4417.182238] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4417.255681] alg: No test for adler32 (adler32-zlib) [ 4418.377362] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 4418.652388] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 4420.327223] Key type lgssc registered [ 4421.982585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4430.913651] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4430.958085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4432.478087] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4436.849617] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4442.601351] LustreError: 152378:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4442.623541] LustreError: 152378:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4446.723611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4447.106992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4451.734458] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4459.493205] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4465.344762] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4472.651456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4472.809226] LustreError: 154203:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4473.137252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4477.295782] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4477.302566] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4480.286915] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4490.317099] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4511.770366] Lustre: lustre-MDT0001: evicting 900889d1-6833-484a-a7fa-5581506380ae (at 192.168.201.10@tcp) by administrative request [ 4511.782794] Lustre: Skipped 1 previous similar message [ 4543.969336] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4543.972441] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4543.975579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4543.986238] Lustre: Skipped 1 previous similar message [ 4549.092416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4549.115547] Lustre: Skipped 2 previous similar messages [ 4554.214294] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4554.223171] Lustre: Skipped 1 previous similar message [ 4556.260065] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4556.389306] Lustre: server umount lustre-OST0000 complete [ 4564.468060] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4564.474161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4564.505802] Lustre: Skipped 1 previous similar message [ 4564.523910] Lustre: Skipped 1 previous similar message [ 4566.692620] Lustre: server umount lustre-MDT0000 complete [ 4569.571806] LustreError: 154226:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4569.596332] LustreError: 154226:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4570.015850] LustreError: 152358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868834 with bad export cookie 7285646826928950280 [ 4570.027252] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4570.031474] LustreError: 152358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4570.230706] Lustre: server umount lustre-MDT0001 complete [ 4582.675648] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 4585.590048] Key type lgssc unregistered [ 4585.907078] LNet: 156452:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4585.921984] LNetError: 156452:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4585.947221] LNet: Removed LNI 192.168.201.110@tcp [ 4587.302110] Key type .llcrypt unregistered [ 4587.309830] Key type ._llcrypt unregistered [ 4672.120688] Key type ._llcrypt registered [ 4672.122853] Key type .llcrypt registered [ 4672.219671] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 4673.258219] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4673.444210] alg: No test for adler32 (adler32-zlib) [ 4674.654053] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 4674.980400] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 4676.705343] Key type lgssc registered [ 4677.967242] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4685.841598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4685.881835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4687.308041] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4692.805329] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4697.578115] LustreError: 158943:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4697.594306] LustreError: 158943:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4702.339976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4702.689036] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4702.956960] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4707.654243] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4715.100707] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4721.798587] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4729.727832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4730.289414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4732.371205] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4738.175711] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4747.449825] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4747.727251] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4763.107501] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4763.107604] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4763.122432] LustreError: Skipped 1 previous similar message [ 4763.143040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4764.428646] Lustre: server umount lustre-OST0000 complete [ 4768.228539] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4768.240692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4768.250913] Lustre: Skipped 2 previous similar messages [ 4768.279362] Lustre: Skipped 2 previous similar messages [ 4773.344745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4773.359479] Lustre: Skipped 1 previous similar message [ 4774.649763] Lustre: server umount lustre-MDT0000 complete [ 4778.340748] LustreError: 158923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869042 with bad export cookie 13174629105800507367 [ 4778.345227] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4778.357081] LustreError: 158923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4778.917703] Lustre: server umount lustre-MDT0001 complete [ 4791.434843] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 4794.558059] Key type lgssc unregistered [ 4794.824812] LNet: 162490:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4794.828643] LNetError: 162490:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4794.844185] LNet: Removed LNI 192.168.201.110@tcp [ 4795.772536] Key type .llcrypt unregistered [ 4795.775877] Key type ._llcrypt unregistered [ 4821.970454] Key type ._llcrypt registered [ 4821.976054] Key type .llcrypt registered [ 4822.129885] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 4823.268139] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4823.485103] alg: No test for adler32 (adler32-zlib) [ 4824.618371] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 4824.826945] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 4826.531907] Key type lgssc registered [ 4827.963581] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4845.101849] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:18:27 (1787869107) [ 4860.855979] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 4869.746946] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 4910.531228] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4910.560639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4911.911896] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4911.977558] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4912.081774] Lustre: lustre-MDT0000: new disk, initializing [ 4912.295238] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4912.331403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4916.977905] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4929.741532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4929.901874] Lustre: 166757:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4929.962587] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4929.970251] Lustre: Skipped 1 previous similar message [ 4930.072341] Lustre: lustre-MDT0001: new disk, initializing [ 4930.139444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4930.185805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4930.200375] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4934.837276] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4944.455697] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4951.337584] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4959.003319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4959.365618] Lustre: lustre-OST0000: new disk, initializing [ 4959.369526] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4959.375969] Lustre: 168724:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4959.470415] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4965.826540] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 4967.496950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4967.504167] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4967.625989] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4976.081548] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4988.396769] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4988.399461] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4988.403270] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4988.412537] LustreError: Skipped 1 previous similar message [ 4988.444621] Lustre: Skipped 1 previous similar message [ 4992.712546] Lustre: server umount lustre-OST0000 complete [ 4996.576453] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4996.585900] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4996.616147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4996.621586] Lustre: Skipped 1 previous similar message [ 4998.625858] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4998.628324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4998.658218] Lustre: Skipped 1 previous similar message [ 5002.844928] Lustre: server umount lustre-MDT0000 complete [ 5006.833415] LustreError: 166749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869270 with bad export cookie 3052997673500494311 [ 5006.848200] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5006.861668] LustreError: 166749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5007.421877] Lustre: server umount lustre-MDT0001 complete [ 5019.155935] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 5022.212942] Key type lgssc unregistered [ 5022.547771] LNet: 170633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5022.559873] LNetError: 170633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5022.582484] LNet: Removed LNI 192.168.201.110@tcp [ 5023.638989] Key type .llcrypt unregistered [ 5023.647138] Key type ._llcrypt unregistered [ 5047.372722] Key type ._llcrypt registered [ 5047.374579] Key type .llcrypt registered [ 5047.466418] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 5048.507253] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5048.671809] alg: No test for adler32 (adler32-zlib) [ 5049.711798] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 5049.918027] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 5051.583132] Key type lgssc registered [ 5052.619470] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5060.290789] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5060.326305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5061.880576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5067.607755] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 5071.330890] LustreError: 172047:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5071.353391] LustreError: 172047:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5076.449863] LustreError: 172048:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5076.920733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5077.496983] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5082.881970] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 5090.486054] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5095.411890] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5101.690598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5102.010262] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 5102.167893] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5106.317425] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5108.459575] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 5116.340431] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5123.666738] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5123.698793] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5123.723433] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5123.742836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.756646] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5124.484427] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5124.501640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5124.510737] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5124.516183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.529511] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5125.394802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.190304] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5126.198794] Lustre: Skipped 1 previous similar message [ 5126.214471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5126.231916] Lustre: Skipped 1 previous similar message [ 5126.238128] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5126.242396] Lustre: Skipped 1 previous similar message [ 5126.246305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.265917] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5126.276194] Lustre: Skipped 1 previous similar message [ 5127.098699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.924397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.722144] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5128.728685] Lustre: Skipped 2 previous similar messages [ 5128.743240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5128.753654] Lustre: Skipped 2 previous similar messages [ 5128.757601] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5128.762485] Lustre: Skipped 2 previous similar messages [ 5128.765806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.772324] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5128.778797] Lustre: Skipped 2 previous similar messages [ 5129.500230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.660773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.966215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.150845] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5133.163842] Lustre: Skipped 3 previous similar messages [ 5133.180877] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5133.190350] Lustre: Skipped 3 previous similar messages [ 5133.194270] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5133.197144] Lustre: Skipped 3 previous similar messages [ 5133.204472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.219602] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5133.228426] Lustre: Skipped 3 previous similar messages [ 5134.297805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.065198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.845405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.542335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.453582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.375402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.515918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.461992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.312411] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5141.316903] Lustre: Skipped 8 previous similar messages [ 5141.330657] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5141.342409] Lustre: Skipped 8 previous similar messages [ 5141.345640] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5141.348063] Lustre: Skipped 8 previous similar messages [ 5141.351375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.360781] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5141.365224] Lustre: Skipped 8 previous similar messages [ 5142.234453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.117631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.920948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.908210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.789858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.985146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.296821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.360217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.303906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.403706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.404929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.454345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.079672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.130309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.049624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.915467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.558075] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5157.561197] Lustre: Skipped 16 previous similar messages [ 5157.569526] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5157.577461] Lustre: Skipped 16 previous similar messages [ 5157.580112] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5157.582943] Lustre: Skipped 16 previous similar messages [ 5157.588495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.596977] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5157.608544] Lustre: Skipped 16 previous similar messages [ 5158.569719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.528529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.718129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.852974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.130620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.299206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.301241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.268824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.219920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.142996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.893780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.897531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.855981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.761571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.540474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.355489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.161043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.000212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.907200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.911900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.746612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.626618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.542198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.519816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.380698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.212212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.058315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.068636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.051855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.247576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.136478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.042668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.909685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.761961] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5189.766787] Lustre: Skipped 33 previous similar messages [ 5189.784228] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5189.791694] Lustre: Skipped 33 previous similar messages [ 5189.796225] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5189.803777] Lustre: Skipped 33 previous similar messages [ 5189.809624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.822789] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5189.825665] Lustre: Skipped 33 previous similar messages [ 5190.718560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.701378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.672480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.697792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.646646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.675809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.468747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.344121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.211943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.348128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.159118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.053254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.896670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.911571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.726450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.728844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.612253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.821093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.842719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.868336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.050179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.813584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.544837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.379204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.569579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.361387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.323840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.174159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.072707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.928200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.807585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.583974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.534102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.439274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.291161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.155975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.265661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.182346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.333211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.390396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.823512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.628994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.623891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.614872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.804378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.624408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.428895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.076381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.925789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.766768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.627209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.308456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.101659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.897517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.902128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.777282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.787917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.642305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.479288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.405417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.229139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.089851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.948440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.894808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.742420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.758641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.644498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.752043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.756426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.720405] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5254.725184] Lustre: Skipped 69 previous similar messages [ 5254.738127] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5254.748565] Lustre: Skipped 69 previous similar messages [ 5254.753541] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5254.756228] Lustre: Skipped 69 previous similar messages [ 5254.760930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.768427] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5254.777193] Lustre: Skipped 69 previous similar messages [ 5255.550953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.480489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.339384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.191207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.051871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.114529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.901195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.772472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.631334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.577420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.406540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.537790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.435785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.297464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.225156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.136614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.115331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.936740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.913434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.978315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.934941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.986300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.802308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.773924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.681026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.598942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.460216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.529444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.335177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.267274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.443288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.506762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.328578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.304932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.478033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.611405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.854582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.849595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.683614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.824427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.103668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.334894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.300508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.338440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.183687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.230299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.114222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.960273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.884508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.753552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.531738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.526877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.600768] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5306.607156] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5306.622037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.386097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.324226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.156798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.051153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.902919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.776543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.728780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.510394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.473512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.297146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.178337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.033675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.023421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.951850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.930463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.897687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.865093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.724481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.005834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.262443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.349789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.277546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.194095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.943491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.886782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.759191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.770096] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5331.782724] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5331.790485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.703740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.823418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.821838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.618967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.500605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.335428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.185734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.041340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.797480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.826309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.762375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.424189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.193775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.167882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.965862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.961317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.868134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.712459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.507724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.475753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.282719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.184706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.011950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.829079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.699192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.539970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.260357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.110630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.923240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.033840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.879919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.892661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.680206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.629948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.777282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.558749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.462674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.106270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.003900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.796942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.604647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.522734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.385556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.208933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.080177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.886727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.614144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.296758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.256256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.170596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.204544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.229330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.048509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.781378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.602614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.373030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.215095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.188091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.056898] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5383.062261] Lustre: Skipped 138 previous similar messages [ 5383.072963] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5383.081775] Lustre: Skipped 138 previous similar messages [ 5383.083993] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5383.088463] Lustre: Skipped 138 previous similar messages [ 5383.091600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.102946] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5383.107827] Lustre: Skipped 138 previous similar messages [ 5383.916862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.615227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.326166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.079656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.848683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.816277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.979522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.049715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.049889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.954069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.002170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.844739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.593106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.525158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.398121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.147833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.826924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.816494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.608707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.410314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.223642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.210340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.059741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.815348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.487339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.334140] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5405.341228] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5405.357640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.127131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.962556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.800224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.776209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.505883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.733754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.637651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.676942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.579722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.552408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.388276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.456703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.364303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.154929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.908466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.728442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.502437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.456759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.257310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.171986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.016259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.868164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.626370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.460579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.177363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.884437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.552310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.351688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.983329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.601368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.255653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.994936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.634201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.366820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.307930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.079221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.214804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.983738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.983776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.761568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.873366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.825033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.815627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.956020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.265426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.100884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.080541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.039253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.005603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.252355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.425602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.558591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.610200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.668611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.610587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.576943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.730277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.963058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.920642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.977303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.914711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.012901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.961874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.221764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.121750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.174811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.192193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.129665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.915974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.067966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.998225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.758726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.522094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.365273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.263054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.047973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.784146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.613671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.391542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.110901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.877588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.696691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.510414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.243485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.995251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.800962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.596412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.437391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.176960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.030848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.852775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.640663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.340442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.147775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.929560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.693276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.345171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.086685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.763583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.578955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.306956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.160349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.861491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.710030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.621067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.627116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.571676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.484928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.504953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.354173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.333718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.197618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.920740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.675785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.444067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.168987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.851361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.556697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.268300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.112068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.048317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.963115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.719149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.602657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.646748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.499348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.311596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.074873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.743882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.487287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.306241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.038905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.080317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.915815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.772708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.825038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.633608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.471537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.394675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.135363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.887464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.766204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.642485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.452768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.344975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.399665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.261085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.186138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.993674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.126550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.027087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.928834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.891688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.783748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.702707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.743344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.648838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.446796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.230803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.067569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.775705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.692332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.403385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.084789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.909143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.760125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.433221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.274566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.173032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.266022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.165591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.222678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.212128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.969653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.712634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.580276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.316068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.079385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.789973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.493824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.427364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.201239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.004203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.713060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.501483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.421152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.131536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.081519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.916381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.927995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.942446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.770520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.828270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.734502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.830129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.773279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.025310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.908410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.882579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.785698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.639726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.639920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.294569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.181470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.025403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.854494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.547731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.304184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.044545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.727844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.432881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.219187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.978814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.848851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.909144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.012908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.894097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.877512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.865708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.675844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.694475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.651793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.453488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.513331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.373495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.332894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.119504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.063569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.873945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.953818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.830064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.804413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.702319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.612413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.585211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.389433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.207678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.992463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.905576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.925080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.819294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.591469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.526789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.351966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.285185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.178593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.221243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.221563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.173351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.246386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.205060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.270170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.144245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.953897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.877161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.880591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.767915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.643706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.497884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.238410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.048115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.869522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.754729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.543321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.498214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.205786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.962016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.695325] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5639.699167] Lustre: Skipped 293 previous similar messages [ 5639.720700] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5639.737304] Lustre: Skipped 293 previous similar messages [ 5639.741852] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5639.748579] Lustre: Skipped 293 previous similar messages [ 5639.752592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.759800] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5639.769251] Lustre: Skipped 293 previous similar messages [ 5640.567711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.361435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.288329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.075651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.813260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.493738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.309921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.055395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.974674] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5646.982486] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5646.999829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.955543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.047169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.072387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.231783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.137057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.968933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.905779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.717850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.581733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.436399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.216625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.027972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.941336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.817919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.345446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.310722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.167853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.470561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.756265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.600429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.460086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.286493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.244208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.157640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.292768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.281996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.398420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.404812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.346773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.266511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.298283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.313961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.161972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.228788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.594545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.829160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.782541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.925943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.475391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.579740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.492390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.519838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.436780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.370734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.069969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.102361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.113681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.190151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.977310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.811454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.878980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.227129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.412553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.387865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.474594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.368942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.034605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.965216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.997536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.000980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.087261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.983619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.063857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.918499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.711689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.638345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.710740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.652160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.681562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.581407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.563270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.324411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.233671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.034694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.974493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.101956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.220404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.024305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.956897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.869264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.814912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.797761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.748512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.556576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.574816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.499919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.376050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.359320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.302384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.247341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.445912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.493213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.568452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.636979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.654865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.636445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.589924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.437855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.146279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.051167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.045404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.210110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.108716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.169738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.125814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.030863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.781530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.766280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.744892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.694582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.724918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.529971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.530032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.434132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.366567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.407445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.283787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.273287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.457446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.432870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.508244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.157407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.068624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.785991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.627431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.354625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.258435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.167668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.049706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.877888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.629894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.544179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.418395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.321231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.233769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.256205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.188863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.113711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.850356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.682818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.532157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.326897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.207301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.181756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.980854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.704100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.497691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.277496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.239169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.961652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.913726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.743900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.602603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.612902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.444427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.315797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.272889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.305656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.325230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.288324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.291830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.330790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.347973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.260422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.265663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.173781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.958310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.882626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.778293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.805314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.598647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.602419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.537588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.625192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.481734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.543931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.457831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.400981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.265628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.155887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.820032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.656956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.411075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.339199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.206795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.164143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.042607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.128715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.290444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.503468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.492861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.579554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.404444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.315621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.084364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.096917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.087416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.307219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.466140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.451859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.577359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.767479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.909778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.153746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.306508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.222518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.452325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.535195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.749277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.783745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.004589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.126409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.399042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.790781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.051636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.268405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.568462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.772849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.862959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.062767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.036937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.045658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.049616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.231439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.192454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.076785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.101739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.406346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.255091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.155710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.074849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.884113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.822811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.971665] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5873.993218] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5873.995476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.784333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.847309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.830191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.924616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.699483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.685913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.480919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.732906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.596921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.806419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.605917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.527295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.666653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.484330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.537052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.818940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.772794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.764600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.514408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.469287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.406964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.533141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.571492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.628440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.453259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.338174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.203424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.196700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.926792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.778456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.745042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.768208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.726914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.646613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.592318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.522845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.556079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.534728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.795111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.863294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.077926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.258463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.362055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.324229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.638409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.910975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.996423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.458816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.776501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.649061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.886539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.898638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.136406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.324618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.510046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.589677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.796072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.807643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.902834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.815881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.887502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.662390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.682627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.890423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.028442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.336842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.462565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.430245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.271348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.419562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.375148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.227037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.328098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.498917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.278687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.068463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.991393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.683078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.612961] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5954.623247] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5954.630339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.459806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.489673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.358462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.421233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.194676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.145741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.991646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.872897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.675595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.697772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.589258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.354861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.175858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.038850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.258053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.124840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.066899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.222393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.146174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.996869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.081951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.940683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.788872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.768437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.799762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.647905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.778443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.695160] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5980.710688] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5980.730944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.657067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.552651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.451469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.489308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.223476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.246924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.996361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.181527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.179280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.157115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.210459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.048568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.857833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.684508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.549243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.703262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.734186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.778388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.761497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.807794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.806759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.712946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.581578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.516422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.610262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.689281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.783885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.964530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.800180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.757749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.625719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.812581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.840964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.082063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.272452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.275423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.330872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.299265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.335751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.279832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.409410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.416187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.241523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.211242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.276390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.451546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.783084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.048117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.294569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.428507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.642358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.744849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.881710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.079995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.341655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.873886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.393976] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6040.407655] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 6040.411113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.417446] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6040.438452] LustreError: 172067:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 6041.555700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.844813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.956682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.964213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.886476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.950153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.067266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.935513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.762568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.839557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.036468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.037311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.052614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.020159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.846574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.063814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.100662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.317635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.474680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.272887] LustreError: 173868:0:(ofd_dev.c:1794:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6417.285653] LustreError: 172067:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6816.740546] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6816.752361] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6816.769828] Lustre: Skipped 426 previous similar messages [ 6816.776646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6818.791324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6818.802318] Lustre: Skipped 2 previous similar messages [ 6820.603356] Lustre: server umount lustre-OST0000 complete [ 6828.512430] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6828.522609] LustreError: Skipped 1 previous similar message [ 6828.535071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6830.828137] Lustre: server umount lustre-MDT0000 complete [ 6834.144565] LustreError: 172048:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6834.161957] LustreError: 172048:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 6834.591376] LustreError: 172027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871098 with bad export cookie 16368805914107900860 [ 6834.601755] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6834.605039] LustreError: 172027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6834.947051] Lustre: server umount lustre-MDT0001 complete [ 6847.734111] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 6850.548992] Key type lgssc unregistered [ 6850.852636] LNet: 227264:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6850.861438] LNetError: 227264:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6850.887994] LNet: Removed LNI 192.168.201.110@tcp [ 6851.817424] Key type .llcrypt unregistered [ 6851.819796] Key type ._llcrypt unregistered [ 6878.784474] Key type ._llcrypt registered [ 6878.786098] Key type .llcrypt registered [ 6878.923454] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 6880.635370] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6880.705531] alg: No test for adler32 (adler32-zlib) [ 6881.983243] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 6882.501738] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 6884.327200] Key type lgssc registered [ 6886.013400] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6895.463381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6895.514741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6897.078825] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6902.437764] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 6907.376080] LustreError: 228682:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6907.414540] LustreError: 228682:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 6911.157705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6911.779734] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6916.607188] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 6923.791773] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6929.370208] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6935.661528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6935.942647] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6942.077955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49664 to 0x280000401:49695) [ 6942.230041] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 6949.551229] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6955.449335] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6957.283169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6957.296174] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6957.301437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.322376] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6957.322692] LustreError: 228702:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6962.536117] Lustre: lustre-OST0000: Not available for connect from 192.168.201.10@tcp (stopping) [ 6962.659937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6962.668431] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6965.758758] Lustre: server umount lustre-OST0000 complete [ 6977.399148] LustreError: 228702:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6999.515025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6999.813990] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7002.870172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.887617] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.897418] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7005.357974] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7012.768401] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7014.099262] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7020.230279] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9d3d4b77c800.ost_server_uuid 50 [ 7021.816661] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9d3d4b77c800.ost_server_uuid in FULL state after 0 sec [ 7027.170338] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7027.181175] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7027.201335] Lustre: Skipped 1 previous similar message [ 7027.206486] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7027.209145] Lustre: Skipped 2 previous similar messages [ 7028.708574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7028.723688] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7028.741977] Lustre: Skipped 1 previous similar message [ 7032.409943] Lustre: server umount lustre-OST0000 complete [ 7038.953866] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7038.973890] Lustre: Skipped 1 previous similar message [ 7038.988869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7038.996283] Lustre: Skipped 1 previous similar message [ 7042.907726] Lustre: server umount lustre-MDT0000 complete [ 7044.068090] LustreError: 228681:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7044.079749] LustreError: 228681:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7046.476191] LustreError: 228663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871310 with bad export cookie 2915185113631829534 [ 7046.479760] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7046.490749] LustreError: 228663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7046.839067] Lustre: server umount lustre-MDT0001 complete [ 7056.731740] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 7059.459567] Key type lgssc unregistered [ 7059.829957] LNet: 233891:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7059.838238] LNetError: 233891:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7060.903991] LNet: Removed LNI 192.168.201.110@tcp [ 7061.990265] Key type .llcrypt unregistered [ 7061.995404] Key type ._llcrypt unregistered [ 7095.494748] Key type ._llcrypt registered [ 7095.497789] Key type .llcrypt registered [ 7095.668929] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 7096.461549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7096.510604] alg: No test for adler32 (adler32-zlib) [ 7097.722492] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 7098.021141] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 7099.815305] Key type lgssc registered [ 7101.122349] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7112.589466] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7112.624866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7114.455070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7120.469885] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7124.962051] LustreError: 235623:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7125.005752] LustreError: 235623:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7130.098104] LustreError: 235624:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7130.744721] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7131.221087] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7136.567396] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7139.903094] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7147.517990] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7148.035487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7154.868788] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7159.361418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49727) [ 7164.512538] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7164.696460] Lustre: lustre-OST0001: new disk, initializing [ 7164.700303] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7164.706850] Lustre: 237934:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7164.766108] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7171.233487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7171.246570] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7171.337817] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7171.397743] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7183.369867] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7193.387823] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7212.515767] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7212.523504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7212.534591] Lustre: Skipped 1 previous similar message [ 7212.551513] Lustre: Skipped 1 previous similar message [ 7216.838556] Lustre: server umount lustre-MDT0000 complete [ 7217.635228] LustreError: 235619:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7217.673285] LustreError: 235619:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 7220.784567] LustreError: 235605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871484 with bad export cookie 8004604946686738581 [ 7220.798949] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7221.240395] Lustre: server umount lustre-MDT0001 complete [ 7235.980595] Lustre: server umount lustre-OST0000 complete [ 7238.111488] Lustre: 234886:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787871486/real 1787871486] req@ffff8ab64f894000 x1874719001380352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787871502 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7238.137366] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7238.157414] Lustre: Skipped 2 previous similar messages [ 7240.252679] Lustre: server umount lustre-OST0001 complete [ 7264.067796] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 7273.914407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7274.193777] LustreError: 241040:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7274.272994] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7278.792268] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7279.589008] LustreError: 241041:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7287.800228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7288.141488] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7293.002745] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7295.890545] Lustre: 242182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7302.667845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7309.583572] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7313.328987] LustreError: 242534:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7313.350608] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7313.351039] LustreError: 242534:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7318.191255] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7318.493482] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7318.506650] Lustre: Skipped 1 previous similar message [ 7319.584948] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49759) [ 7324.715236] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7332.539683] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7336.327363] Lustre: 244054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7346.380131] Lustre: server umount mylustre-MDT0000 complete [ 7349.743985] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7349.760207] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7349.789347] LustreError: 241040:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7349.878200] LustreError: 241015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871613 with bad export cookie 8004604946686741444 [ 7349.892636] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7349.896288] LustreError: 241015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 7350.258711] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7350.266451] Lustre: Skipped 1 previous similar message [ 7352.312805] Lustre: server umount mylustre-MDT0001 complete [ 7356.778563] Lustre: server umount mylustre-OST0000 complete [ 7360.745628] Lustre: server umount mylustre-OST0001 complete [ 7382.213263] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 7393.499128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7393.937191] LustreError: 246361:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7393.957335] LustreError: 246361:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7394.042138] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7399.557828] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7409.478266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7415.067436] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7417.955789] Lustre: 247513:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7426.434780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7434.057550] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 7434.392782] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7438.156333] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49791) [ 7443.890430] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7449.592663] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7450.295860] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7457.602846] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7461.619453] Lustre: 249388:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7470.069261] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7470.078621] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7470.087934] Lustre: Skipped 7 previous similar messages [ 7470.101205] Lustre: Skipped 2 previous similar messages [ 7474.668786] Lustre: server umount tfs-MDT0000 complete [ 7475.174749] LustreError: 246356:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7475.201469] LustreError: 246356:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 12 previous similar messages [ 7478.972241] LustreError: 246337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871742 with bad export cookie 8004604946686743397 [ 7478.976929] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7478.985308] LustreError: 246337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7479.683316] Lustre: server umount tfs-MDT0001 complete [ 7494.896651] Lustre: server umount tfs-OST0000 complete [ 7496.354067] Lustre: 234886:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787871744/real 1787871744] req@ffff8ab746bea1c0 x1874719001493888/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787871760 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7496.460392] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7499.745270] Lustre: 234884:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787871747/real 1787871747] req@ffff8ab64f895a40 x1874719001494144/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787871763 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7500.176194] Lustre: server umount tfs-OST0001 complete [ 7525.928867] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 7537.795740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7538.357674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7538.361179] Lustre: Skipped 3 previous similar messages [ 7542.944547] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7553.461247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7559.740284] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7562.465239] Lustre: 252780:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7570.677956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7577.122928] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7578.385320] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 7583.485987] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49823) [ 7587.498789] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7592.969128] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7595.950889] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7605.166129] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7609.297625] Lustre: 254650:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7615.960707] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:04:38 (1787871878) [ 7633.907410] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7633.920213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7633.927298] Lustre: Skipped 1 previous similar message [ 7633.953648] Lustre: Skipped 3 previous similar messages [ 7639.020818] LustreError: 251634:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7639.041704] LustreError: 251634:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 9 previous similar messages [ 7639.085496] Lustre: server umount lustre-MDT0000 complete [ 7643.374296] LustreError: 251614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871907 with bad export cookie 8004604946686745308 [ 7643.380001] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7643.387859] LustreError: 251614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7644.150305] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7644.172838] Lustre: Skipped 1 previous similar message [ 7650.045244] Lustre: server umount lustre-MDT0001 complete [ 7660.716676] Lustre: server umount lustre-OST0000 complete [ 7670.997912] Lustre: server umount lustre-OST0001 complete [ 7678.341760] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 7688.566601] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 7731.767929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7732.138723] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7732.168493] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7732.241179] Lustre: lustre-MDT0000: new disk, initializing [ 7732.313422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7732.318143] Lustre: Skipped 3 previous similar messages [ 7732.334934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7737.558798] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7751.060258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7751.303709] Lustre: 257838:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7751.541845] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7751.544612] Lustre: Skipped 1 previous similar message [ 7751.841852] Lustre: lustre-MDT0001: new disk, initializing [ 7752.145827] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7752.170251] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7758.331676] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7771.918075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7772.205827] Lustre: lustre-OST0000: new disk, initializing [ 7772.218656] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7772.225896] Lustre: 259423:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7773.541874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7773.565452] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7773.805312] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7780.279755] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7792.406348] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7792.605952] Lustre: lustre-OST0001: new disk, initializing [ 7792.609821] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7792.614649] Lustre: 260294:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7794.617086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7794.633748] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7794.765087] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7799.842254] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 7812.662885] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:07:55 (1787872075) [ 7827.452362] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:08:10 (1787872090) [ 7845.345758] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7845.360778] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7845.361680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7845.383865] Lustre: Skipped 5 previous similar messages [ 7845.405410] Lustre: Skipped 3 previous similar messages [ 7849.031799] Lustre: server umount lustre-OST0000 complete [ 7854.561208] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7854.564776] LustreError: Skipped 1 previous similar message [ 7866.348386] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7866.356268] Lustre: Skipped 9 previous similar messages [ 7867.359281] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7867.717646] Lustre: server umount lustre-MDT0000 complete [ 7871.507397] LustreError: 258644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872135 with bad export cookie 8004604946686747093 [ 7871.511324] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7871.516887] LustreError: 258644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7877.953143] Lustre: server umount lustre-MDT0001 complete [ 7890.877875] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 7962.094241] Lustre: server umount lustre-OST0001 complete [ 7968.760617] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 7983.174521] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 8020.166938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8020.482422] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8020.517355] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8020.623344] Lustre: lustre-MDT0000: new disk, initializing [ 8020.709889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8020.715606] Lustre: Skipped 3 previous similar messages [ 8020.726358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8026.020219] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8037.653165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8037.834610] Lustre: 265140:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8037.894902] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8037.900883] Lustre: Skipped 1 previous similar message [ 8037.996442] Lustre: lustre-MDT0001: new disk, initializing [ 8038.273990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8038.291916] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8043.095373] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8053.162186] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8059.727528] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8068.424699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8068.738903] Lustre: 267106:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8069.861918] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8070.041751] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8075.078944] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8084.726584] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8095.717172] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8095.718383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8095.720943] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8095.726526] Lustre: Skipped 5 previous similar messages [ 8095.751124] Lustre: Skipped 2 previous similar messages [ 8097.938226] Lustre: server umount lustre-OST0000 complete [ 8104.928246] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8104.934913] LustreError: Skipped 1 previous similar message [ 8108.310699] Lustre: server umount lustre-MDT0000 complete [ 8111.078076] LustreError: 265146:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8111.116110] LustreError: 265146:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 15 previous similar messages [ 8112.534089] LustreError: 268118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872376 with bad export cookie 8004604946686748556 [ 8112.545476] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8112.550473] LustreError: 268118:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8113.144845] Lustre: server umount lustre-MDT0001 complete [ 8124.149398] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 8126.622411] Key type lgssc unregistered [ 8126.907352] LNet: 268872:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8126.913291] LNetError: 268872:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8126.928986] LNet: Removed LNI 192.168.201.110@tcp [ 8127.746212] Key type .llcrypt unregistered [ 8127.750272] Key type ._llcrypt unregistered [ 8158.941179] Key type ._llcrypt registered [ 8158.946642] Key type .llcrypt registered [ 8159.199736] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 8160.377224] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8160.497581] alg: No test for adler32 (adler32-zlib) [ 8161.628691] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 8161.925385] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 8163.695223] Key type lgssc registered [ 8165.149045] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8174.228949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8174.264757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8175.777424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8180.372879] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8185.337070] LustreError: 270336:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8185.354110] LustreError: 270336:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8188.477183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8188.975615] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8193.674651] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8200.787370] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8207.309511] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8215.988665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8216.690266] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8220.823693] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8224.173804] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8233.623493] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8234.631766] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8236.524803] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8236.530639] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8236.557899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8238.577902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8238.579972] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8238.599755] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8238.610966] Lustre: Skipped 1 previous similar message [ 8242.692019] Lustre: server umount lustre-OST0000 complete [ 8248.802882] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8248.812724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8248.825148] Lustre: Skipped 1 previous similar message [ 8252.981101] Lustre: server umount lustre-MDT0000 complete [ 8253.925818] LustreError: 271065:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8253.948978] LustreError: 271065:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8256.598552] LustreError: 270317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872520 with bad export cookie 13268427401000523618 [ 8256.615302] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8256.616250] LustreError: 270317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8256.978594] Lustre: server umount lustre-MDT0001 complete [ 8267.605528] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 8270.675357] Key type lgssc unregistered [ 8270.930722] LNet: 273924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8270.940154] LNetError: 273924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8270.953670] LNet: Removed LNI 192.168.201.110@tcp [ 8271.782431] Key type .llcrypt unregistered [ 8271.786372] Key type ._llcrypt unregistered [ 8290.319874] Key type ._llcrypt registered [ 8290.329060] Key type .llcrypt registered [ 8290.465487] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 8292.239310] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8292.334096] alg: No test for adler32 (adler32-zlib) [ 8293.378987] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 8293.768894] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 8295.511921] Key type lgssc registered [ 8296.413761] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8305.358227] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8305.381436] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8320.991433] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8326.111329] LustreError: 275121:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.110@tcp: failed processing log, type 4: rc = -110 [ 8352.801341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8358.416540] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8364.934675] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8366.986973] Lustre: server umount lustre-OST0000 complete [ 8382.657935] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 8385.428532] Key type lgssc unregistered [ 8385.804214] LNet: 276755:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8385.813768] LNetError: 276755:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8385.835710] LNet: Removed LNI 192.168.201.110@tcp [ 8386.815907] Key type .llcrypt unregistered [ 8386.817792] Key type ._llcrypt unregistered [ 8429.949486] Key type ._llcrypt registered [ 8429.960109] Key type .llcrypt registered [ 8430.122383] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 8431.432209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8431.488754] alg: No test for adler32 (adler32-zlib) [ 8432.713729] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 8433.006555] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 8434.767174] Key type lgssc registered [ 8436.424496] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8451.224412] loop0: detected capacity change from 0 to 204800000 [ 8451.443775] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8451.459015] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8451.882889] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8452.994278] loop0: detected capacity change from 0 to 204800000 [ 8453.226517] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8453.244881] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8453.677995] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8454.869881] loop0: detected capacity change from 0 to 204800000 [ 8455.040370] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8455.060714] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8455.174608] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8456.374837] loop0: detected capacity change from 0 to 204800000 [ 8456.566892] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8456.583455] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8456.731660] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.275525] loop0: detected capacity change from 0 to 204800000 [ 8458.324470] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8463.591176] loop0: detected capacity change from 0 to 204800000 [ 8463.666320] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8468.383693] loop0: detected capacity change from 0 to 204800000 [ 8468.430393] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8473.885088] loop0: detected capacity change from 0 to 204800000 [ 8473.924617] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8478.544746] loop0: detected capacity change from 0 to 204800000 [ 8478.630451] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8478.690252] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8478.753449] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8478.760161] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8478.766659] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8478.785506] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8478.797477] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8478.817119] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8478.828455] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8478.833528] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8478.839561] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8478.845582] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8478.861532] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8478.873281] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8478.888832] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8478.924683] Lustre: 280174:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8486.240388] Lustre: server umount MGS complete [ 8487.184507] loop0: detected capacity change from 0 to 204800000 [ 8487.296341] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8487.685098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8487.745753] Lustre: lustre-MDT0000: Aborting client recovery [ 8488.734741] loop1: detected capacity change from 0 to 204800000 [ 8488.879591] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8489.081364] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8489.111570] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8489.125899] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8489.144421] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8489.159116] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8489.172715] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8489.184683] Lustre: 280607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8489.222775] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8489.390549] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8489.452708] Lustre: lustre-MDT0001: Aborting client recovery [ 8490.321676] loop2: detected capacity change from 0 to 204800000 [ 8490.477417] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8490.549360] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8490.558436] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8490.564950] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8490.570837] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8490.577532] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8490.586985] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8490.596309] Lustre: 280687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8490.684063] Lustre: Found index 0 for lustre-OST0000, updating log [ 8490.711389] Lustre: 280705:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8490.755025] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8490.779448] Lustre: lustre-OST0000: Aborting recovery [ 8491.659946] loop3: detected capacity change from 0 to 204800000 [ 8491.839507] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8491.922258] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8491.947028] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8491.959937] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8491.978730] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8491.993560] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8492.002934] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8492.025948] Lustre: 280769:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8492.049715] Lustre: Found index 1 for lustre-OST0001, updating log [ 8492.090334] Lustre: 280774:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8492.173098] Lustre: lustre-OST0001: Aborting recovery [ 8497.319095] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8497.328916] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8497.338199] LustreError: 280551:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8497.392194] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8497.442280] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8499.376392] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8499.381883] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8500.174554] LustreError: 280518:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1677us: rc = -2 [ 8500.678223] LustreError: 281010:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 352us: rc = -2 [ 8500.700329] LustreError: 281010:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 27 previous similar messages [ 8515.039923] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8515.048905] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8515.050348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8515.084909] Lustre: Skipped 2 previous similar messages [ 8519.917953] Lustre: server umount lustre-MDT0000 complete [ 8520.160909] LustreError: 280521:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8520.201314] LustreError: 280521:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [ 8521.075885] LustreError: 280502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872785 with bad export cookie 17214684454935177300 [ 8521.086822] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8521.095587] LustreError: 280502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8525.283258] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8525.284092] LustreError: 280516:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8525.285348] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8525.285357] Lustre: Skipped 3 previous similar messages [ 8525.299094] Lustre: Skipped 2 previous similar messages [ 8525.345803] LustreError: 280516:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8526.303955] LustreError: 280515:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8526.306505] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8526.327204] Lustre: Skipped 2 previous similar messages [ 8527.509398] Lustre: server umount lustre-MDT0001 complete [ 8534.647083] Lustre: server umount lustre-OST0000 complete [ 8549.855163] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8550.191028] Lustre: server umount lustre-OST0001 complete [ 8557.572665] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 19:20:20 (1787872820) [ 8585.222223] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 8595.503175] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 8631.590759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8631.807036] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8631.824046] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8631.893129] Lustre: lustre-MDT0000: new disk, initializing [ 8631.962187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8631.972652] Lustre: Skipped 1 previous similar message [ 8632.000263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8635.968201] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8645.759221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8645.929351] Lustre: 284846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8645.974408] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8645.978532] Lustre: Skipped 1 previous similar message [ 8646.051112] Lustre: lustre-MDT0001: new disk, initializing [ 8646.141611] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8646.181304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8646.188359] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8650.510538] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8661.346425] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8668.495842] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8677.498589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8677.806690] Lustre: lustre-OST0000: new disk, initializing [ 8677.810909] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8677.822852] Lustre: 286814:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8677.939852] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8684.094832] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8686.686986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8686.697321] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8686.836682] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8693.024933] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8698.408764] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8711.985479] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8722.086635] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8727.746282] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8768.995108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8769.005606] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8769.027413] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8769.035356] Lustre: Skipped 1 previous similar message [ 8774.116372] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8774.121396] Lustre: Skipped 1 previous similar message [ 8779.231145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8779.512932] Lustre: server umount lustre-OST0000 complete [ 8784.364035] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8784.364063] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8784.381030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8784.381039] Lustre: Skipped 2 previous similar messages [ 8784.391338] Lustre: Skipped 2 previous similar messages [ 8784.418170] LustreError: Skipped 1 previous similar message [ 8789.202472] Lustre: server umount lustre-MDT0000 complete [ 8789.477394] LustreError: 284853:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8789.510811] LustreError: 284853:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8792.431312] LustreError: 284837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873056 with bad export cookie 17214684454935180009 [ 8792.442112] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8792.941571] Lustre: server umount lustre-MDT0001 complete [ 8800.733918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8804.870478] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8809.559504] Lustre: server umount MGS complete [ 8820.510928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8820.848496] LustreError: 290824:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8820.888781] LustreError: 290837:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8820.966756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8824.735762] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8830.452544] LustreError: 290824:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8830.471958] LustreError: 290824:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8832.676219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8838.849583] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8846.991569] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8853.146860] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8859.883525] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8860.145993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8860.159790] Lustre: Skipped 1 previous similar message [ 8866.726778] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 8873.795063] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8901.088855] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8901.096849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8901.112706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8901.117572] Lustre: Skipped 1 previous similar message [ 8901.591232] Lustre: server umount lustre-OST0000 complete [ 8909.286107] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8909.302436] Lustre: Skipped 1 previous similar message [ 8911.400848] Lustre: server umount lustre-MDT0000 complete [ 8914.404027] LustreError: 290825:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8914.428193] LustreError: 290825:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8915.023974] LustreError: 293550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873179 with bad export cookie 17214684454935181850 [ 8915.035266] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8915.043953] LustreError: 293550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8925.849442] Lustre: server umount lustre-MDT0001 complete [ 8936.166447] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 8939.284105] Key type lgssc unregistered [ 8939.670613] LNet: 294773:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8939.677765] LNetError: 294773:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8940.713549] LNet: Removed LNI 192.168.201.110@tcp [ 8941.889617] Key type .llcrypt unregistered [ 8941.891431] Key type ._llcrypt unregistered [ 8982.487446] Key type ._llcrypt registered [ 8982.490242] Key type .llcrypt registered [ 8982.627834] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 8998.669610] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 8999.710332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8999.874600] alg: No test for adler32 (adler32-zlib) [ 9000.904876] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 9001.217563] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 9002.879176] Key type lgssc registered [ 9004.005336] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9039.156914] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9039.180620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9040.529210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9040.577930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9040.699797] Lustre: lustre-MDT0000: new disk, initializing [ 9040.805562] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9040.833688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9045.767361] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9055.149468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9055.287213] Lustre: 298734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9055.338261] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9055.342485] Lustre: Skipped 1 previous similar message [ 9055.429209] Lustre: lustre-MDT0001: new disk, initializing [ 9055.555385] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9055.611930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9055.647155] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9059.866481] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9069.227883] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9075.544543] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9082.753145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9083.033778] Lustre: lustre-OST0000: new disk, initializing [ 9083.037277] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9083.047473] Lustre: 300702:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9083.141123] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9088.605333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9088.611127] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9088.726388] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9089.076827] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9097.189868] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9104.271506] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9116.049585] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9124.803566] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9176.035627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9176.048261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9176.050492] Lustre: Skipped 1 previous similar message [ 9176.058215] Lustre: Skipped 1 previous similar message [ 9181.162561] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9181.172687] Lustre: Skipped 1 previous similar message [ 9185.247176] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9185.551638] Lustre: server umount lustre-OST0000 complete [ 9191.392445] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9191.396152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9191.420833] Lustre: Skipped 1 previous similar message [ 9191.439434] Lustre: Skipped 1 previous similar message [ 9196.595162] Lustre: server umount lustre-MDT0000 complete [ 9200.256071] LustreError: 298725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873464 with bad export cookie 17609913714225078246 [ 9200.265470] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9200.278456] LustreError: 298725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9200.611896] Lustre: server umount lustre-MDT0001 complete [ 9209.701449] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9214.305516] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9218.501515] Lustre: server umount MGS complete [ 9229.303041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9229.655553] LustreError: 304618:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9229.678626] LustreError: 304618:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 9229.687596] LustreError: 304631:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9229.740057] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9233.565568] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9234.913305] LustreError: 304619:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9239.010870] LustreError: 304618:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9241.145757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9241.395342] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9245.740145] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9252.110576] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9257.943686] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9264.681630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9265.130636] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9271.993799] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9279.238557] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9307.105995] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9307.115697] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9307.125791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9310.189603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9310.214676] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9313.165422] Lustre: server umount lustre-OST0000 complete [ 9318.368446] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9318.375377] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9318.389603] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9318.398063] Lustre: Skipped 2 previous similar messages [ 9330.661410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9330.670705] Lustre: Skipped 5 previous similar messages [ 9331.172289] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9331.382631] Lustre: server umount lustre-MDT0000 complete [ 9335.017878] LustreError: 304600:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873599 with bad export cookie 17609913714225080087 [ 9335.023158] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9335.026794] LustreError: 304600:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9335.417056] Lustre: server umount lustre-MDT0001 complete [ 9346.873430] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [ 9349.881138] Key type lgssc unregistered [ 9350.114711] LNet: 308568:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9350.119157] LNetError: 308568:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9351.142754] LNet: Removed LNI 192.168.201.110@tcp [ 9352.341309] Key type .llcrypt unregistered [ 9352.344842] Key type ._llcrypt unregistered [ 9378.795801] Key type ._llcrypt registered [ 9378.799892] Key type .llcrypt registered [ 9378.897408] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [ 9397.147347] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 9398.227875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9398.368927] alg: No test for adler32 (adler32-zlib) [ 9399.555389] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [ 9399.816427] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [ 9401.519121] Key type lgssc registered [ 9402.633680] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9427.662178] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [ 9442.005538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9442.039533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9443.321895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9443.357804] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9443.452673] Lustre: lustre-MDT0000: new disk, initializing [ 9443.544408] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9443.568718] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9448.349636] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [ 9459.444123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9459.624239] Lustre: lustre-OST0000: new disk, initializing [ 9459.626498] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9459.629069] Lustre: Skipped 1 previous similar message [ 9459.631944] Lustre: 312620:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9459.681722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9459.962308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9459.980292] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9460.021384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9466.210783] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10252.849862] Lustre: server umount lustre-MDT0000 complete [10257.396803] LustreError: 311808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787874521 with bad export cookie 12569446759036630767 [10257.410959] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10267.774842] Lustre: server umount lustre-OST0000 complete [10298.245468] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [10310.641719] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [10348.458316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10348.746402] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10348.778893] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10348.869286] Lustre: lustre-MDT0000: new disk, initializing [10349.016882] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10349.048299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10354.271961] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10365.064545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10365.189710] Lustre: 316949:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10365.238339] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10365.241904] Lustre: Skipped 1 previous similar message [10365.310925] Lustre: lustre-MDT0001: new disk, initializing [10365.421250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10365.504886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10365.517691] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10369.938719] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10379.500399] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10385.466469] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10392.179934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10392.494230] Lustre: lustre-OST0000: new disk, initializing [10392.498443] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10392.514606] Lustre: 318919:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10392.650959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10399.189969] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10400.350894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10400.363937] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10400.487704] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10408.730957] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10413.960837] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10414.243738] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10419.527989] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10419.801574] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10426.336366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10426.340574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10426.358241] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10426.406132] Lustre: Skipped 1 previous similar message [10428.088212] Lustre: server umount lustre-OST0000 complete [10436.577673] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10436.587801] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10436.598284] Lustre: Skipped 1 previous similar message [10436.616679] Lustre: Skipped 2 previous similar messages [10439.062547] Lustre: server umount lustre-MDT0000 complete [10443.662207] LustreError: 317746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787874707 with bad export cookie 12569446759039571579 [10443.664176] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10443.670080] LustreError: 317746:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10444.087977] Lustre: server umount lustre-MDT0001 complete [10448.364215] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10450.806249] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:51:52 (1787874712) [10459.541033] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10460.185845] LustreError: 321233:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10460.213616] LustreError: 321233:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10460.340713] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10465.760408] LustreError: 321234:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10466.601980] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10470.882398] LustreError: 321233:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10476.000709] LustreError: 321234:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10478.691608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10479.308288] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10485.299921] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10495.183278] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10501.882583] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10510.363461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10510.872394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10518.756758] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10528.349098] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10535.568424] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10535.800728] Lustre: lustre-OST0001: new disk, initializing [10535.804565] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10535.810514] Lustre: 324070:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10535.959713] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10542.182804] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10543.800594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10543.821957] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10543.939930] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10543.958798] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10545.006786] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10545.020531] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10546.096379] LustreError: 324503:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10546.117257] LustreError: 324503:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10548.145891] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10548.156914] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10552.292705] LustreError: 324503:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10552.317349] LustreError: 324503:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10552.823407] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10553.099599] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10559.963940] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9d3d45409800.ost_server_uuid 50 [10560.553137] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10560.568465] LustreError: 324497:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10561.733528] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9d3d45409800.ost_server_uuid in FULL state after 0 sec [10567.405602] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9d3d45409800.ost_server_uuid 50 [10568.873750] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9d3d45409800.ost_server_uuid in FULL state after 0 sec [10570.995355] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10585.075898] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10585.092524] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10585.100672] Lustre: Skipped 1 previous similar message [10589.086411] Lustre: lustre-OST0001: Not available for connect from 192.168.201.10@tcp (stopping) [10590.878859] Lustre: server umount lustre-OST0001 complete [10594.150960] LustreError: 323057:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.10@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10600.448058] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10600.459643] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10600.472417] Lustre: Skipped 2 previous similar messages [10600.494959] Lustre: Skipped 3 previous similar messages [10605.355555] Lustre: server umount lustre-OST0000 complete [10610.657422] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10610.663450] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10610.673437] Lustre: Skipped 1 previous similar message [10610.683051] Lustre: Skipped 1 previous similar message [10615.785739] LustreError: 322348:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10615.820287] LustreError: 322348:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [10615.989608] Lustre: server umount lustre-MDT0000 complete [10620.458439] LustreError: 321214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787874884 with bad export cookie 12569446759039572468 [10620.463769] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10620.476459] LustreError: 321214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10620.904997] Lustre: server umount lustre-MDT0001 complete [10633.011225] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [10635.838807] Key type lgssc unregistered [10636.332707] LNet: 326764:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10636.340253] LNetError: 326764:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10636.366845] LNet: Removed LNI 192.168.201.110@tcp [10637.168930] Key type .llcrypt unregistered [10637.170325] Key type ._llcrypt unregistered [10662.276658] Key type ._llcrypt registered [10662.279678] Key type .llcrypt registered [10662.404693] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [10664.078383] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10664.121188] alg: No test for adler32 (adler32-zlib) [10665.117873] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [10665.316899] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [10666.992130] Key type lgssc registered [10668.505043] Lustre: Echo OBD driver; http://www.lustre.org/ [10676.594431] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10676.610554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10678.024064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10683.254843] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10687.463332] LustreError: 328187:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10687.506223] LustreError: 328187:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10690.820437] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10692.578345] LustreError: 328186:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10697.698472] LustreError: 328187:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10698.148719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10698.620856] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10703.528932] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10713.232273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10713.628989] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10720.656756] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10722.871253] LustreError: 329864:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10722.915372] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10728.732720] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10728.951318] LustreError: 329865:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10728.980227] LustreError: 329865:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10736.222450] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10736.429234] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10741.761232] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10743.248829] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10750.402189] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10757.991688] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10758.214694] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10859.491080] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10859.502831] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10859.512236] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10859.522531] Lustre: Skipped 1 previous similar message [10864.619735] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10864.633949] Lustre: Skipped 1 previous similar message [10869.732137] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10869.740846] Lustre: Skipped 1 previous similar message [10871.264497] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10871.349652] Lustre: server umount lustre-OST0001 complete [10874.870564] LustreError: 329866:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10874.881838] LustreError: 329866:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10879.993234] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10880.008513] Lustre: Skipped 1 previous similar message [10880.054773] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10880.084451] Lustre: Skipped 1 previous similar message [10884.617451] Lustre: server umount lustre-OST0000 complete [10891.444955] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10891.457205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10891.467619] Lustre: Skipped 1 previous similar message [10891.513622] Lustre: Skipped 1 previous similar message [10898.731609] Lustre: server umount lustre-MDT0000 complete [10900.960955] LustreError: 328181:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10900.989013] LustreError: 328181:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10904.061244] LustreError: 328167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787875168 with bad export cookie 6747619377342762821 [10904.075586] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10904.659549] Lustre: server umount lustre-MDT0001 complete [10917.803443] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [10920.787713] Key type lgssc unregistered [10921.050801] LNet: 333134:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10921.060733] LNetError: 333134:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10922.086482] LNet: Removed LNI 192.168.201.110@tcp [10922.918055] Key type .llcrypt unregistered [10922.922387] Key type ._llcrypt unregistered [10966.995465] Key type ._llcrypt registered [10966.997285] Key type .llcrypt registered [10967.094208] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [10968.008932] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10968.081319] alg: No test for adler32 (adler32-zlib) [10969.385400] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [10969.987534] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [10971.841401] Key type lgssc registered [10973.308854] Lustre: Echo OBD driver; http://www.lustre.org/ [10980.861107] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10980.875203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10982.190978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10987.197359] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [10992.612180] LustreError: 335426:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10992.625681] LustreError: 335426:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10997.751595] LustreError: 335425:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10998.892027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10999.694900] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11006.359563] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11010.505419] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11017.645488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11018.067981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11024.427920] LustreError: 337138:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11027.505613] LustreError: 336923:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11027.529438] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11027.538920] LustreError: 336923:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11027.771788] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11032.555298] LustreError: 336923:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11032.587703] LustreError: 336923:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11041.332024] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11041.628726] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11046.937344] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11050.269308] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11061.584461] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11074.365760] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11091.939205] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11091.945650] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11091.965767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11092.965083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11092.967092] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11093.004469] Lustre: Skipped 1 previous similar message [11095.062046] Lustre: server umount lustre-MDT0000 complete [11098.084032] LustreError: 335421:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11098.103650] LustreError: 335421:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [11098.951178] LustreError: 335407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787875362 with bad export cookie 5005210834288608161 [11098.960254] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11098.966137] LustreError: 335407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11099.482395] Lustre: server umount lustre-MDT0001 complete [11112.608508] Lustre: server umount lustre-OST0000 complete [11126.630600] Lustre: server umount lustre-OST0001 complete [11142.942813] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [11153.793291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11154.562401] LustreError: 340705:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11154.657189] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11159.906853] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11168.769857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11169.150982] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11175.396674] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11178.933991] Lustre: 341846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11186.529775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11187.030477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11194.191517] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11194.215923] LustreError: 342198:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11194.237285] LustreError: 342198:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11199.365323] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11203.463500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11209.208265] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11210.425114] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11218.896612] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11222.525895] Lustre: 343717:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11229.673502] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11229.689925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11229.690310] Lustre: Skipped 4 previous similar messages [11229.709457] Lustre: Skipped 6 previous similar messages [11233.629086] Lustre: server umount lustre-MDT0000 complete [11237.059978] LustreError: 340685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787875501 with bad export cookie 5005210834288609862 [11237.067381] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11237.498262] Lustre: server umount lustre-MDT0001 complete [11252.001204] Lustre: server umount lustre-OST0000 complete [11255.263143] Lustre: 334689:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787875503/real 1787875503] req@ffff8ab6431192c0 x1874723060443008/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787875519 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11255.291924] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11255.969635] Lustre: server umount lustre-OST0001 complete [11260.117052] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11261.745747] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11263.504559] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:05:26 (1787875526) [11286.716870] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [11295.310622] loop0: detected capacity change from 0 to 1099511627776 [11295.919155] SGI XFS with ACLs, security attributes, quota, no debug enabled [11295.926746] *** VALIDATE xfs *** [11295.936837] XFS (loop0): Mounting V5 Filesystem [11295.961234] XFS (loop0): Ending clean mount [11297.666535] loop1: detected capacity change from 0 to 18691697672192 [11411.394300] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [11420.633810] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [11454.333661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11454.504527] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11454.524990] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11454.593983] Lustre: lustre-MDT0000: new disk, initializing [11454.711184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11454.713954] Lustre: Skipped 1 previous similar message [11454.738125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11458.872720] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11468.056387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11468.169338] Lustre: 348852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11468.231259] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11468.234311] Lustre: Skipped 1 previous similar message [11468.289917] Lustre: lustre-MDT0001: new disk, initializing [11468.399049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11468.407485] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11472.340689] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11480.241110] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11484.795846] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11490.391233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11490.725865] Lustre: lustre-OST0000: new disk, initializing [11490.728843] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11490.734755] Lustre: 350821:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11492.602537] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11492.616383] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11492.786685] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11496.246921] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11504.734541] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11508.951533] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11509.133629] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11513.483924] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11513.669269] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11518.435379] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11518.435792] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11518.439982] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11518.446972] Lustre: Skipped 1 previous similar message [11518.453611] Lustre: Skipped 1 previous similar message [11521.639349] Lustre: server umount lustre-OST0000 complete [11528.687992] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11528.699764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11532.313268] Lustre: server umount lustre-MDT0000 complete [11533.798161] LustreError: 348862:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11533.821416] LustreError: 348862:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [11535.894409] LustreError: 348846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787875799 with bad export cookie 5005210834288611416 [11535.899666] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11535.906162] LustreError: 348846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [11536.222156] Lustre: server umount lustre-MDT0001 complete [11544.759846] XFS (loop0): Unmounting Filesystem [11550.843868] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:10:12 (1787875812) [11560.837347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11561.311616] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11561.319581] Lustre: Skipped 2 previous similar messages [11565.806557] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11573.762653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11579.128610] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11587.272306] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11593.112673] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11599.865171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11606.320847] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11613.486980] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11630.052547] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11630.057066] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11630.058550] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11630.058556] Lustre: Skipped 1 previous similar message [11630.092504] Lustre: Skipped 2 previous similar messages [11635.523359] Lustre: server umount lustre-OST0000 complete [11645.296144] Lustre: server umount lustre-MDT0000 complete [11648.742544] LustreError: 355228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787875912 with bad export cookie 5005210834288612298 [11648.759758] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11648.760677] LustreError: 355228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11649.122183] Lustre: server umount lustre-MDT0001 complete [11660.226893] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [11663.127921] Key type lgssc unregistered [11663.454839] LNet: 357037:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11663.463338] LNetError: 357037:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11664.485827] LNet: Removed LNI 192.168.201.110@tcp [11665.446606] Key type .llcrypt unregistered [11665.453333] Key type ._llcrypt unregistered [11694.739265] Key type ._llcrypt registered [11694.742387] Key type .llcrypt registered [11694.846843] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [11695.998162] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11696.279828] alg: No test for adler32 (adler32-zlib) [11697.449571] Lustre: Lustre: Build Version: 2.17.57_106_g3758560 [11697.667523] LNet: Added LNI 192.168.201.110@tcp [8/256/0/180] [11699.326064] Key type lgssc registered [11700.571955] Lustre: Echo OBD driver; http://www.lustre.org/ [11709.050981] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11709.089954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11710.680404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11716.426972] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11720.175976] LustreError: 358611:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11720.211829] LustreError: 358611:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11725.292534] LustreError: 358612:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11730.321899] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11730.411215] LustreError: 358611:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11730.773661] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11737.752215] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11747.321638] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11753.814936] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11759.985496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11760.447385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11768.053287] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11777.837102] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11791.330623] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11791.333294] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11791.350245] Lustre: Skipped 1 previous similar message [11791.375562] Lustre: Skipped 1 previous similar message [11794.309059] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.10@tcp (stopping) [11796.046386] Lustre: server umount lustre-MDT0001 complete [11796.461596] LustreError: 358608:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11796.469670] LustreError: 358608:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11801.579981] LustreError: 358612:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11801.613204] LustreError: 358612:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11809.609857] LustreError: 358606:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.10@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11809.628103] LustreError: 358606:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [11813.258753] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11813.343365] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11813.347706] Lustre: Found index 1 for lustre-MDT0001, updating log [11813.401437] Lustre: 358598:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11813.909158] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11818.993324] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11820.438197] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11822.087174] LustreError: 362129:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11822.102229] LustreError: 362129:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11822.118290] LustreError: 358603:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11822.223603] LustreError: 362135:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.110@tcp: cfg command failed: rc = -17 [11822.247919] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11822.247919] [11834.336955] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11834.339731] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11834.366332] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11834.376431] Lustre: Skipped 1 previous similar message [11838.454719] Lustre: server umount lustre-MDT0001 complete [11839.473304] LustreError: 358611:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11839.499856] LustreError: 358611:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [11850.490862] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11850.656069] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11850.658193] Lustre: Found index 1 for lustre-MDT0001, updating log [11850.708493] Lustre: 358598:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11851.250633] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11856.370320] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11856.481243] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [11860.472475] LustreError: 363288:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11860.478300] LustreError: 363288:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11860.489827] LustreError: 358603:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11860.494217] LustreError: 358603:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11860.545592] LustreError: 363291:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11860.559973] LustreError: 363291:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11860.565120] LustreError: 363291:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.110@tcp: cfg command failed: rc = -17 [11860.571451] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.110@tcp [11860.571451] [12163.930196] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.10@tcp (stopping) [12163.944748] Lustre: Skipped 1 previous similar message [12168.702416] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12168.706899] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12168.722319] Lustre: Skipped 1 previous similar message [12168.744588] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12168.761871] Lustre: Skipped 1 previous similar message [12170.198236] Lustre: server umount lustre-MDT0001 complete [12173.807889] LustreError: 365921:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12173.842079] LustreError: 365921:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 8 previous similar messages [12181.423129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12181.522721] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12181.532573] Lustre: Found index 1 for lustre-MDT0001, updating log [12181.589991] Lustre: 358601:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12182.092431] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12185.864597] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [12187.143515] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12190.708076] LustreError: 371973:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12190.712082] LustreError: 371973:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12190.716167] LustreError: 358603:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12190.719849] LustreError: 358603:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12190.781661] LustreError: 371976:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12190.791652] LustreError: 371976:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.110@tcp: cfg command failed: rc = -114 [12190.795331] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12190.795331] [12507.626832] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12507.632851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12507.642435] Lustre: Skipped 2 previous similar messages [12507.650145] Lustre: Skipped 3 previous similar messages [12513.700343] Lustre: server umount lustre-MDT0000 complete [12517.873351] LustreError: 365921:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12517.883300] LustreError: 365921:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [12518.606867] LustreError: 358592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787876782 with bad export cookie 15846156571141639020 [12518.611665] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12518.633962] LustreError: 358592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12522.976925] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12524.001901] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12524.019857] Lustre: Skipped 4 previous similar messages [12525.231259] Lustre: server umount lustre-MDT0001 complete [12529.953721] Lustre: server umount lustre-OST0000 complete [12541.688367] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [12551.771529] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [12587.048580] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:27:29 (1787876849) [12594.156965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12594.393194] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12594.414685] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12594.502858] Lustre: lustre-MDT0000: new disk, initializing [12594.590623] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12594.608567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12599.544808] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [12610.011434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12610.120569] Lustre: 381943:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12610.212563] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12610.216946] Lustre: Skipped 1 previous similar message [12610.316884] Lustre: lustre-MDT0001: new disk, initializing [12610.434871] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12610.496708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12610.519551] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12615.210907] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [12626.162136] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12633.418305] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12640.848246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12641.223294] Lustre: lustre-OST0000: new disk, initializing [12641.227866] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12641.238036] Lustre: 383917:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12641.349883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12649.055729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12649.070397] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12649.175322] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12649.254361] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_default_debug -1 all [12661.065785] Lustre: DEBUG MARKER: oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12669.920467] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12669.923568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12669.978351] Lustre: Skipped 2 previous similar messages [12670.002504] Lustre: Skipped 2 previous similar messages [12675.757867] Lustre: server umount lustre-MDT0000 complete [12680.163555] LustreError: 381955:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12680.187416] LustreError: 381955:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [12680.224435] LustreError: 382752:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12680.228067] LustreError: 383916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787876944 with bad export cookie 15846156571141640966 [12680.236226] LustreError: MGC192.168.201.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12680.275575] LustreError: 383916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12680.620872] Lustre: server umount lustre-MDT0001 complete [12697.061760] Lustre: server umount lustre-OST0000 complete [12737.595915] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing set_hostid [12753.521823] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing load_modules_local [12792.048616] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12492 sec ========= 20:30:54 (1787877054) [12794.166732] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:30:56 (1787877056) === [12798.825158] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:31:01 (1787877061) === [12852.802161] Lustre: DEBUG MARKER: oleg110-server.virtnet: executing unload_modules_local [12856.409490] Key type lgssc unregistered [12856.781553] LNet: 390315:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12856.794376] LNetError: 390315:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12856.819208] LNet: Removed LNI 192.168.201.110@tcp [12858.340260] Key type .llcrypt unregistered [12858.346109] Key type ._llcrypt unregistered