[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 492164808 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2841992K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002232] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004012] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009007] pid_max: default: 32768 minimum: 301 [ 0.011117] LSM: Security Framework initializing [ 0.012054] Yama: becoming mindful. [ 0.013039] SELinux: Initializing. [ 0.014082] *** VALIDATE selinux *** [ 0.022738] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026567] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027168] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028112] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029125] *** VALIDATE tmpfs *** [ 0.030460] *** VALIDATE proc *** [ 0.032127] *** VALIDATE cgroup *** [ 0.033013] *** VALIDATE cgroup2 *** [ 0.034281] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036156] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038036] Spectre V2 : User space: Vulnerable [ 0.039011] Speculative Store Bypass: Vulnerable [ 0.042155] debug: unmapping init [mem 0xffffffff9f259000-0xffffffff9f260fff] [ 0.045000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045855] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046022] ... version: 2 [ 0.047013] ... bit width: 48 [ 0.047954] ... generic registers: 4 [ 0.048015] ... value mask: 0000ffffffffffff [ 0.049017] ... max period: 00007fffffffffff [ 0.050016] ... fixed-purpose events: 3 [ 0.051035] ... event mask: 000000070000000f [ 0.052351] rcu: Hierarchical SRCU implementation. [ 0.054561] smp: Bringing up secondary CPUs ... [ 0.055693] x86: Booting SMP configuration: [ 0.056027] .... node #0, CPUs: #1 #2 #3 [ 0.060381] smp: Brought up 1 node, 4 CPUs [ 0.062018] smpboot: Max logical packages: 1 [ 0.063019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.138690] node 0 deferred pages initialised in 72ms [ 0.142125] devtmpfs: initialized [ 0.143230] x86/mm: Memory block size: 128MB [ 0.145809] gcov: version magic: 0x41383552 [ 0.146531] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.150094] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.153270] pinctrl core: initialized pinctrl subsystem [ 0.155176] [ 0.155805] ************************************************************* [ 0.158019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.160012] ** ** [ 0.163015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.165011] ** ** [ 0.167014] ** This means that this kernel is built to expose internal ** [ 0.169012] ** IOMMU data structures, which may compromise security on ** [ 0.172015] ** your system. ** [ 0.174012] ** ** [ 0.177014] ** If you see this message and you are not debugging the ** [ 0.179013] ** kernel, report this immediately to your vendor! ** [ 0.182016] ** ** [ 0.184012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.187013] ************************************************************* [ 0.189721] NET: Registered protocol family 16 [ 0.191437] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.194066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.196062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.200115] cpuidle: using governor menu [ 0.201668] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.204406] PCI: Using configuration type 1 for base access [ 0.206096] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.216053] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.218116] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.222155] cryptd: max_cpu_qlen set to 1000 [ 0.225253] ACPI: Added _OSI(Module Device) [ 0.227030] ACPI: Added _OSI(Processor Device) [ 0.230019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.232024] ACPI: Added _OSI(Processor Aggregator Device) [ 0.236517] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.243422] ACPI: Interpreter enabled [ 0.245094] ACPI: PM: (supports S0 S3 S4 S5) [ 0.247034] ACPI: Using IOAPIC for interrupt routing [ 0.249119] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.252401] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.264065] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.266040] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.269019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.272093] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.278105] acpiphp: Slot [2] registered [ 0.280192] acpiphp: Slot [5] registered [ 0.281161] acpiphp: Slot [6] registered [ 0.283145] acpiphp: Slot [7] registered [ 0.284116] acpiphp: Slot [8] registered [ 0.286149] acpiphp: Slot [9] registered [ 0.288133] acpiphp: Slot [10] registered [ 0.289168] acpiphp: Slot [3] registered [ 0.291106] acpiphp: Slot [4] registered [ 0.292102] acpiphp: Slot [11] registered [ 0.294202] acpiphp: Slot [12] registered [ 0.296117] acpiphp: Slot [13] registered [ 0.298121] acpiphp: Slot [14] registered [ 0.299111] acpiphp: Slot [15] registered [ 0.301146] acpiphp: Slot [16] registered [ 0.303151] acpiphp: Slot [17] registered [ 0.305142] acpiphp: Slot [18] registered [ 0.307121] acpiphp: Slot [19] registered [ 0.309116] acpiphp: Slot [20] registered [ 0.311098] acpiphp: Slot [21] registered [ 0.313116] acpiphp: Slot [22] registered [ 0.314109] acpiphp: Slot [23] registered [ 0.316152] acpiphp: Slot [24] registered [ 0.318101] acpiphp: Slot [25] registered [ 0.320115] acpiphp: Slot [26] registered [ 0.322112] acpiphp: Slot [27] registered [ 0.323099] acpiphp: Slot [28] registered [ 0.325169] acpiphp: Slot [29] registered [ 0.326084] acpiphp: Slot [30] registered [ 0.327117] acpiphp: Slot [31] registered [ 0.329086] PCI host bridge to bus 0000:00 [ 0.331022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.334034] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.337030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.339034] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.342030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.343049] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.346215] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.350110] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.354438] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.364015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.368938] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.373025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.375018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.378050] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.381445] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.384024] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.387041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.389805] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.396016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.409959] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.415017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.422720] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.430016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.438018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.470016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.484186] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.491016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.499016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.521026] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.531616] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.541018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.547014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.562016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.573611] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.583017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.593018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.611018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.621272] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.629029] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.637028] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.662023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.676699] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.685107] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.693016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.708020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.722903] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.725470] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.728460] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.731497] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.734435] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.741071] iommu: Default domain type: Passthrough [ 0.743459] SCSI subsystem initialized [ 0.744149] ACPI: bus type USB registered [ 0.746351] usbcore: registered new interface driver usbfs [ 0.748113] usbcore: registered new interface driver hub [ 0.750096] usbcore: registered new device driver usb [ 0.753221] pps_core: LinuxPPS API ver. 1 registered [ 0.755014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.759072] PTP clock support registered [ 0.761169] EDAC MC: Ver: 3.0.0 [ 0.764226] PCI: Using ACPI for IRQ routing [ 0.767114] NetLabel: Initializing [ 0.768016] NetLabel: domain hash size = 128 [ 0.770016] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.772125] NetLabel: unlabeled traffic allowed by default [ 0.775074] vgaarb: loaded [ 0.777189] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.779017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.786362] clocksource: Switched to clocksource kvm-clock [ 0.892478] VFS: Disk quotas dquot_6.6.0 [ 0.894930] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.897693] *** VALIDATE ramfs *** [ 0.899061] *** VALIDATE hugetlbfs *** [ 0.900668] pnp: PnP ACPI init [ 0.903077] pnp: PnP ACPI: found 6 devices [ 0.923661] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.929462] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.932101] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.934966] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.937800] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.940802] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.943818] NET: Registered protocol family 2 [ 0.946543] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.952029] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.956512] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.962350] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.966247] TCP: Hash tables configured (established 65536 bind 65536) [ 0.969623] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.973271] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.976607] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.979977] NET: Registered protocol family 1 [ 0.982693] RPC: Registered named UNIX socket transport module. [ 0.984976] RPC: Registered udp transport module. [ 0.986842] RPC: Registered tcp transport module. [ 0.988756] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.990995] NET: Registered protocol family 44 [ 0.992475] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.994622] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.996894] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.999636] PCI: CLS 0 bytes, default 64 [ 1.001247] Unpacking initramfs... [ 2.413690] debug: unmapping init [mem 0xffffa0de7cc54000-0xffffa0de7ffbffff] [ 2.417675] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.422141] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.425407] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.935126] Initialise system trusted keyrings [ 2.937120] Key type blacklist registered [ 2.939158] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.952279] zbud: loaded [ 2.956253] *** VALIDATE nfs *** [ 2.957914] *** VALIDATE nfs4 *** [ 2.959865] pstore: using deflate compression [ 2.962528] Platform Keyring initialized [ 3.093468] NET: Registered protocol family 38 [ 3.095161] Key type asymmetric registered [ 3.096434] Asymmetric key parser 'x509' registered [ 3.098393] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.101848] io scheduler mq-deadline registered [ 3.104799] io scheduler kyber registered [ 3.106926] io scheduler bfq registered [ 3.109686] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.113265] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.116691] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.119542] ACPI: Power Button [PWRF] [ 3.125095] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.132346] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.150696] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.161945] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.183631] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.212645] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.241890] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.247514] Non-volatile memory driver v1.3 [ 3.249309] Linux agpgart interface v0.103 [ 3.280893] virtio_blk virtio1: [vda] 150640 512-byte logical blocks (77.1 MB/73.6 MiB) [ 3.284208] vda: detected capacity change from 0 to 77127680 [ 3.297696] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.300415] vdb: detected capacity change from 0 to 1073741824 [ 3.314511] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.317871] vdc: detected capacity change from 0 to 2621440000 [ 3.331395] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.334813] vdd: detected capacity change from 0 to 2621440000 [ 3.349056] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.351797] vde: detected capacity change from 0 to 4294967296 [ 3.366187] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.369311] vdf: detected capacity change from 0 to 4294967296 [ 3.377051] libphy: Fixed MDIO Bus: probed [ 3.384227] usbcore: registered new interface driver usbserial_generic [ 3.387843] usbserial: USB Serial support registered for generic [ 3.390895] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.394191] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.395392] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.398078] mousedev: PS/2 mouse device common for all mice [ 3.401364] rtc_cmos 00:05: RTC can wake from S4 [ 3.404733] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.406168] rtc_cmos 00:05: registered as rtc0 [ 3.410563] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.414260] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.415548] intel_pstate: CPU model not supported [ 3.419818] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.423386] hid: raw HID events driver (C) Jiri Kosina [ 3.424989] usbcore: registered new interface driver usbhid [ 3.426341] usbhid: USB HID core driver [ 3.427533] drop_monitor: Initializing network drop monitor service [ 3.429193] Initializing XFRM netlink socket [ 3.430680] NET: Registered protocol family 10 [ 3.433887] Segment Routing with IPv6 [ 3.435518] NET: Registered protocol family 17 [ 3.438110] mpls_gso: MPLS GSO support [ 3.444244] RAS: Correctable Errors collector initialized. [ 3.446875] AVX version of gcm_enc/dec engaged. [ 3.448901] AES CTR mode by8 optimization enabled [ 3.522574] sched_clock: Marking stable (3522540002, 0)->(4437363800, -914823798) [ 3.526318] registered taskstats version 1 [ 3.528451] Loading compiled-in X.509 certificates [ 3.530648] zswap: loaded using pool lzo/zbud [ 3.556742] Key type big_key registered [ 3.569851] Key type encrypted registered [ 3.572024] ima: No TPM chip found, activating TPM-bypass! [ 3.573465] ima: Allocated hash algorithm: sha1 [ 3.575281] ima: No architecture policies found [ 3.577148] evm: Initialising EVM extended attributes: [ 3.579387] evm: security.selinux [ 3.580725] evm: security.ima [ 3.581853] evm: security.capability [ 3.583383] evm: HMAC attrs: 0x1 [ 3.586098] rtc_cmos 00:05: setting system clock to 2026-09-30 03:35:48 UTC (1790739348) [ 3.592663] debug: unmapping init [mem 0xffffffffa0203000-0xffffffffa03fffff] [ 3.596164] debug: unmapping init [mem 0xffffffff9ef82000-0xffffffff9f258fff] [ 3.605152] Write protecting the kernel read-only data: 28672k [ 3.608897] debug: unmapping init [mem 0xffffffff9d603000-0xffffffff9d7fffff] [ 3.612147] debug: unmapping init [mem 0xffffffff9df14000-0xffffffff9dffffff] [ 3.655847] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.665748] systemd[1]: Detected virtualization kvm. [ 3.667876] systemd[1]: Detected architecture x86-64. [ 3.669468] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.697862] systemd[1]: No hostname configured. [ 3.699793] systemd[1]: Set hostname to . [ 3.702119] random: systemd: uninitialized urandom read (16 bytes read) [ 3.704904] systemd[1]: Initializing machine ID from random generator. [ 3.744196] random: ln: uninitialized urandom read (6 bytes read) [ 3.846487] random: systemd: uninitialized urandom read (16 bytes read) [ 3.849311] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.855112] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.859741] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.494822] device-mapper: uevent: version 1.0.3 [ 4.497055] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.179853] virtio_net virtio0 ens2: renamed from eth0 [ 5.217199] random: fast init done [ 5.284096] scsi host0: ata_piix [ 5.307201] scsi host1: ata_piix [ 5.308832] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.311404] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.530704] dracut-initqueue[587]: RTNETLINK answers: File exists [ 10.094308] random: crng init done [ 10.095756] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.468770] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.558165] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.822135] SELinux: Disabled at runtime. [ 11.878815] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.888195] systemd[1]: Detected virtualization kvm. [ 11.890266] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.399697] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.403330] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.407837] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.413328] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.418433] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.427723] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.431918] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ 12.463818] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Local Encrypted Volumes. Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. Mounting Huge Pages File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.827511] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.130932] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.175646] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.274918] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.283730] EDAC sbridge: Ver: 1.1.2 [ 14.987045] Key type dns_resolver registered [ 15.292444] NFS: Registering the id_resolver key type [ 15.294419] Key type id_resolver registered [ 15.295955] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg436-server login: [ 41.182387] libcfs: loading out-of-tree module taints kernel. [ 41.203728] Key type ._llcrypt registered [ 41.206129] Key type .llcrypt registered [ 41.254838] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 50.798458] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 51.547364] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 51.556173] alg: No test for adler32 (adler32-zlib) [ 52.583069] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 52.957195] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 54.600205] Key type lgssc registered [ 55.331793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 69.338107] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 114.401383] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 128.719604] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 128.766552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 130.139466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 130.163638] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 130.273640] Lustre: lustre-MDT0000: new disk, initializing [ 130.415376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 130.442075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 135.642179] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 151.019587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 151.123645] Lustre: 6483:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 151.161654] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 151.167307] Lustre: Skipped 1 previous similar message [ 151.280183] Lustre: lustre-MDT0001: new disk, initializing [ 151.378778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 151.406764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 151.419166] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 156.787915] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 162.325518] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 174.580625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 174.909532] Lustre: lustre-OST0000: new disk, initializing [ 174.920230] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 174.931840] Lustre: 8419:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 175.023741] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 180.272006] hrtimer: interrupt took 1998097 ns [ 182.813712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 182.829655] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 182.917645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 183.164404] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 200.526686] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 200.714962] Lustre: lustre-OST0001: new disk, initializing [ 200.720742] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 200.729752] Lustre: 9495:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 200.822143] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 207.646698] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.903356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 207.913885] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 208.011917] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 219.856771] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 227.232970] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 235.120366] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing check_logdir /tmp/testlogs/ [ 240.802048] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing yml_node [ 245.201306] Lustre: DEBUG MARKER: Client: 2.17.59.3 [ 248.028747] Lustre: DEBUG MARKER: MDS: 2.17.59.3 [ 250.673193] Lustre: DEBUG MARKER: OSS: 2.17.59.3 [ 252.352737] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 29 23:39:55 EDT 2026 [ 271.256836] Lustre: DEBUG MARKER: only running tests: 76-120 [ 272.999409] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 274.485647] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 294.889578] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 294.893187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 294.900987] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 294.915756] Lustre: Skipped 1 previous similar message [ 294.951833] Lustre: Skipped 3 previous similar messages [ 296.892874] Lustre: server umount lustre-MDT0000 complete [ 300.932053] LustreError: 8430:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739645 with bad export cookie 16180241418716162085 [ 300.933439] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 300.956075] LustreError: 8430:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 305.122968] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 305.126319] LustreError: 6489:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 305.129189] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 305.145304] Lustre: Skipped 3 previous similar messages [ 305.159036] LustreError: 6489:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 310.243324] LustreError: 6488:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 310.243722] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 310.266874] LustreError: 6488:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 310.293339] Lustre: Skipped 2 previous similar messages [ 311.268630] LustreError: 6489:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 311.288437] LustreError: 6489:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 315.360613] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 315.685680] Lustre: server umount lustre-MDT0001 complete [ 320.401987] Lustre: server umount lustre-OST0000 complete [ 324.150534] Lustre: server umount lustre-OST0001 complete [ 330.882047] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 343.132518] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 386.674373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 386.983374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 387.006863] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 387.087586] Lustre: lustre-MDT0000: new disk, initializing [ 387.199310] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 387.220283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 391.749764] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 405.896327] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 405.986325] Lustre: 15101:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 406.054267] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 406.059174] Lustre: Skipped 1 previous similar message [ 406.101272] Lustre: lustre-MDT0001: new disk, initializing [ 406.187085] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 406.251568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 406.269274] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 412.387629] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 424.034832] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 430.784873] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 438.967459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 439.413333] Lustre: lustre-OST0000: new disk, initializing [ 439.437275] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 439.455452] Lustre: 17071:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 439.643690] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 440.615057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 440.637806] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 440.825930] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 446.301581] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 457.351636] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 462.489492] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 462.716945] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 467.632881] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 467.836594] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 471.527223] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 471.528231] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 471.536743] LustreError: Skipped 1 previous similar message [ 471.537529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 471.537534] Lustre: Skipped 2 previous similar messages [ 471.591288] Lustre: Skipped 1 previous similar message [ 476.097687] Lustre: server umount lustre-OST0000 complete [ 481.767692] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 481.775742] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 481.790636] Lustre: Skipped 1 previous similar message [ 481.810571] Lustre: Skipped 2 previous similar messages [ 486.855095] Lustre: server umount lustre-MDT0000 complete [ 486.885634] LustreError: 15108:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 486.906614] LustreError: 15108:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 491.360683] LustreError: 15092:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739836 with bad export cookie 16180241418716163618 [ 491.373185] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 492.017516] Lustre: server umount lustre-MDT0001 complete [ 498.732761] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 23:44:01 (1790739841) [ 507.492419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 507.994096] LustreError: 19337:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 508.061252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 514.350338] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 517.604869] LustreError: 19338:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.635381] LustreError: 19338:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 526.803632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 531.760707] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 538.956835] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 545.604387] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 553.854536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 562.553512] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 571.531456] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 587.722495] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 599.010773] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 599.026674] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 599.049218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 600.940600] Lustre: server umount lustre-MDT0000 complete [ 604.643659] LustreError: 19334:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 604.660818] LustreError: 19334:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 605.101550] LustreError: 19317:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739950 with bad export cookie 16180241418716164500 [ 605.105108] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 605.118310] LustreError: 19317:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 605.506768] Lustre: server umount lustre-MDT0001 complete [ 620.130706] Lustre: server umount lustre-OST0000 complete [ 649.047739] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 662.826683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 663.208120] LustreError: 23756:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 663.261772] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 663.272371] Lustre: Skipped 2 previous similar messages [ 668.818990] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 678.619440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 683.514121] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 686.902606] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 694.346722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 702.221416] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 712.192756] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 712.385834] Lustre: lustre-OST0001: new disk, initializing [ 712.389176] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 712.394947] Lustre: 26075:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 714.337519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 714.356094] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 714.483207] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 718.559494] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 729.927336] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 734.475201] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 750.567669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 750.590233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 750.596291] Lustre: Skipped 5 previous similar messages [ 750.596572] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 750.633639] Lustre: Skipped 5 previous similar messages [ 753.794944] Lustre: server umount lustre-MDT0000 complete [ 755.690940] LustreError: 23753:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 755.745134] LustreError: 23753:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 759.344673] LustreError: 23736:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740104 with bad export cookie 16180241418716165340 [ 759.356062] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 759.357967] LustreError: 23736:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 759.575912] Lustre: server umount lustre-MDT0001 complete [ 774.849472] Lustre: server umount lustre-OST0000 complete [ 776.160868] Lustre: 3616:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790740105/real 1790740105] req@ffffa0ddc72f6680 x1877726354203264/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790740121 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 776.177053] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 779.048322] Lustre: server umount lustre-OST0001 complete [ 797.806629] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 809.398877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 810.190056] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 810.195185] Lustre: Skipped 3 previous similar messages [ 815.307053] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 825.771637] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 831.683966] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 834.889962] Lustre: 30241:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 842.879260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 849.997570] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 860.314360] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 868.185540] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 876.465755] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 881.073375] Lustre: 32120:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 886.767953] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 886.787924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 886.801552] Lustre: Skipped 2 previous similar messages [ 886.809163] Lustre: Skipped 3 previous similar messages [ 893.101456] Lustre: server umount lustre-MDT0000 complete [ 896.995216] LustreError: 29094:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 897.035445] LustreError: 29094:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 10 previous similar messages [ 897.439554] LustreError: 29075:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740242 with bad export cookie 16180241418716166642 [ 897.447214] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 897.450547] LustreError: 29075:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 897.931625] Lustre: server umount lustre-MDT0001 complete [ 911.435267] Lustre: server umount lustre-OST0000 complete [ 925.903476] Lustre: server umount lustre-OST0001 complete [ 935.475100] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 23:51:18 (1790740278) [ 963.761803] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 973.254382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 978.010329] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 986.164412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 991.263940] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 994.316577] Lustre: 36075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1001.124313] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1008.860799] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1018.601341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1025.595340] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1033.754444] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1037.622238] Lustre: 37958:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1045.476892] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1045.495372] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1045.499726] Lustre: Skipped 4 previous similar messages [ 1045.517361] Lustre: Skipped 7 previous similar messages [ 1049.344405] Lustre: server umount lustre-MDT0000 complete [ 1053.792628] LustreError: 36077:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740398 with bad export cookie 16180241418716167748 [ 1053.796551] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1053.812484] LustreError: 36077:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1054.336889] Lustre: server umount lustre-MDT0001 complete [ 1069.287563] Lustre: server umount lustre-OST0000 complete [ 1071.904245] Lustre: 3618:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790740400/real 1790740400] req@ffffa0defe68b800 x1877726354305408/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790740416 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1074.151737] Lustre: server umount lustre-OST0001 complete [ 1085.155789] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 23:53:47 (1790740427) [ 1103.947656] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1115.647775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1116.364410] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1116.369387] Lustre: Skipped 7 previous similar messages [ 1121.311936] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1131.901191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1138.424334] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1141.923291] Lustre: 41248:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1149.808175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1157.437641] LustreError: 41603:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1157.472214] LustreError: 41603:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 13 previous similar messages [ 1157.800946] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1167.441157] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1175.087320] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1183.627322] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1187.537118] Lustre: 43128:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1188.552749] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1188.561727] Lustre: Skipped 1 previous similar message [ 1193.530401] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.36@tcp (stopping) [ 1193.954604] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1193.961076] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1193.974934] Lustre: Skipped 1 previous similar message [ 1198.010813] Lustre: server umount lustre-MDT0000 complete [ 1202.511408] LustreError: 40080:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740547 with bad export cookie 16180241418716168854 [ 1202.524466] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1209.171512] Lustre: server umount lustre-MDT0001 complete [ 1219.101613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1227.254535] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xe08bc3d7fea3c6fd [ 1227.270406] Lustre: MGC192.168.204.136@tcp: Connection restored to 0@lo (at 0@lo) [ 1232.967079] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1238.004312] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1242.200841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1247.115909] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1247.747425] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1247.754760] Lustre: Skipped 1 previous similar message [ 1255.369825] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1257.345427] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1264.077667] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1265.778315] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1273.355447] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1276.558381] Lustre: server umount lustre-MDT0000 complete [ 1280.561921] LustreError: 41631:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740625 with bad export cookie 16180241418716169981 [ 1280.575278] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1280.575957] LustreError: 41631:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1280.901347] Lustre: server umount lustre-MDT0001 complete [ 1295.744172] Lustre: server umount lustre-OST0000 complete [ 1299.622954] Lustre: 3615:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790740628/real 1790740628] req@ffffa0def7e0a300 x1877726354396032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790740644 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1299.984466] Lustre: server umount lustre-OST0001 complete [ 1309.943924] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 23:57:32 (1790740652) [ 1330.238084] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1342.909118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1348.895628] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1359.206596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1364.960909] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1369.077520] Lustre: 48889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1376.312499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1383.908680] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1392.852188] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1400.201543] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1410.103225] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1416.654487] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1429.830572] Lustre: server umount lustre-MDT0000 complete [ 1434.777495] LustreError: 50775:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740779 with bad export cookie 16180241418716170779 [ 1434.786493] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1434.792375] LustreError: 50775:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1435.044147] Lustre: server umount lustre-MDT0001 complete [ 1449.963824] Lustre: server umount lustre-OST0000 complete [ 1464.856651] Lustre: server umount lustre-OST0001 complete [ 1476.165609] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 00:00:18 (1790740818) [ 1478.951968] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1481.263484] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 00:00:24 (1790740824) [ 1493.361419] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 1503.027207] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1530.370374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1530.617835] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1530.713538] Lustre: lustre-MDT0000: new disk, initializing [ 1530.791657] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1536.415655] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1547.205279] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1554.674398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1555.070888] Lustre: lustre-OST0000: new disk, initializing [ 1555.077968] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1555.083246] Lustre: Skipped 1 previous similar message [ 1555.092511] Lustre: 54981:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1557.166312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1557.176957] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1557.222322] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1562.463434] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1573.372519] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1576.021190] Lustre: DEBUG MARKER: create test files [ 1618.404263] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1618.420833] Lustre: Skipped 14 previous similar messages [ 1618.425683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1618.439039] Lustre: Skipped 18 previous similar messages [ 1621.080774] Lustre: server umount lustre-OST0000 complete [ 1626.395480] Lustre: server umount lustre-MDT0000 complete [ 1640.131596] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1643.260657] Key type lgssc unregistered [ 1643.625830] LNet: 56612:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1643.640872] LNetError: 56612:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1644.711903] LNet: Removed LNI 192.168.204.136@tcp [ 1645.855048] Key type .llcrypt unregistered [ 1645.860358] Key type ._llcrypt unregistered [ 1678.302460] Key type ._llcrypt registered [ 1678.305758] Key type .llcrypt registered [ 1678.486040] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1679.804781] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1680.022714] alg: No test for adler32 (adler32-zlib) [ 1681.112420] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 1681.394984] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1683.080199] Key type lgssc registered [ 1684.399797] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1692.308950] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1692.337074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1693.929568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1698.954135] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1708.200794] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1715.369842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1715.725311] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1718.777329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1718.900860] Lustre: 59362:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1718.908946] Lustre: 59362:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1718.915047] Lustre: 59362:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1718.925866] Lustre: 59362:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1718.938571] Lustre: 59362:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1718.948675] Lustre: 59362:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1724.075109] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1735.263250] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1741.114320] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1747.831319] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1759.716305] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1759.718461] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1759.744173] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1762.871354] Lustre: server umount lustre-OST0000 complete [ 1766.814105] Lustre: server umount lustre-MDT0000 complete [ 1782.807728] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1786.238432] Key type lgssc unregistered [ 1786.681939] LNet: 60971:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1786.690554] LNetError: 60971:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1786.729653] LNet: Removed LNI 192.168.204.136@tcp [ 1788.035444] Key type .llcrypt unregistered [ 1788.038594] Key type ._llcrypt unregistered [ 1821.388616] Key type ._llcrypt registered [ 1821.390219] Key type .llcrypt registered [ 1821.601815] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1823.640598] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1823.704388] alg: No test for adler32 (adler32-zlib) [ 1824.769996] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 1825.136885] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1826.904318] Key type lgssc registered [ 1828.522014] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1838.043727] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1838.076385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1839.655632] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1844.928624] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1854.693747] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1863.016093] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1863.448933] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1864.517956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1864.663449] Lustre: 63820:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1864.689032] Lustre: 63820:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1864.703588] Lustre: 63820:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1864.714536] Lustre: 63820:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1864.732253] Lustre: 63820:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1864.758605] Lustre: 63820:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1871.990516] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1881.369879] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1887.845738] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1900.513211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1900.518453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1900.538609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1905.179658] Lustre: server umount lustre-OST0000 complete [ 1910.047731] Lustre: server umount lustre-MDT0000 complete [ 1923.869662] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1927.354942] Key type lgssc unregistered [ 1927.643101] LNet: 65376:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1927.651688] LNetError: 65376:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1927.665506] LNet: Removed LNI 192.168.204.136@tcp [ 1928.605417] Key type .llcrypt unregistered [ 1928.608828] Key type ._llcrypt unregistered [ 1948.930177] Key type ._llcrypt registered [ 1948.933323] Key type .llcrypt registered [ 1949.024625] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 1965.871354] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1967.081224] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1967.191890] alg: No test for adler32 (adler32-zlib) [ 1968.328120] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 1968.691231] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1970.440305] Key type lgssc registered [ 1971.977677] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2015.335982] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2015.381266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2016.688981] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2016.761860] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2016.874144] Lustre: lustre-MDT0000: new disk, initializing [ 2016.976046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2017.008085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2022.747507] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2036.315086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2036.517916] Lustre: 68558:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2036.608376] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2036.616250] Lustre: Skipped 1 previous similar message [ 2036.720939] Lustre: lustre-MDT0001: new disk, initializing [ 2036.866828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2036.904864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2036.933939] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2041.804498] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2054.149662] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2061.829372] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2071.197145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2071.618082] Lustre: lustre-OST0000: new disk, initializing [ 2071.628404] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2071.644994] Lustre: 70528:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2071.819969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2079.829618] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2079.848963] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2080.086413] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2080.110127] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2092.883731] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2098.297719] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2098.574923] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2104.998195] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2105.242072] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2110.948057] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2110.973439] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2111.011256] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2111.021867] Lustre: Skipped 1 previous similar message [ 2113.756752] Lustre: server umount lustre-OST0000 complete [ 2118.631640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2118.642488] LustreError: Skipped 1 previous similar message [ 2118.648970] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2118.665618] Lustre: Skipped 1 previous similar message [ 2118.677646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2121.191102] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2121.209082] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2124.996429] Lustre: server umount lustre-MDT0000 complete [ 2129.886343] LustreError: 68552:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741474 with bad export cookie 8666334262286202648 [ 2129.917988] LustreError: 68552:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2129.923335] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2130.607069] Lustre: server umount lustre-MDT0001 complete [ 2141.742682] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 00:11:24 (1790741484) [ 2157.626688] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2205.129283] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 2216.796978] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2258.957348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2259.385688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2259.412281] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2259.572933] Lustre: lustre-MDT0000: new disk, initializing [ 2259.737502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2259.784285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2265.531083] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2278.425032] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.537912] Lustre: 76574:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2278.699040] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2278.710477] Lustre: Skipped 1 previous similar message [ 2278.857953] Lustre: lustre-MDT0001: new disk, initializing [ 2278.957755] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2279.011425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2279.028530] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2283.984208] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2294.435922] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2299.870334] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2308.076366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2308.291151] Lustre: lustre-OST0000: new disk, initializing [ 2308.297425] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2308.302489] Lustre: 78546:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2308.397980] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2314.839570] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2314.851767] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2315.025224] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2315.728634] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2326.927503] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2332.421287] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2332.819675] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2338.089191] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2338.389959] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2340.833795] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2340.834461] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2340.845810] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2340.845819] Lustre: Skipped 1 previous similar message [ 2340.856102] LustreError: Skipped 1 previous similar message [ 2340.863793] Lustre: Skipped 1 previous similar message [ 2345.956815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2345.964413] Lustre: Skipped 2 previous similar messages [ 2346.411114] Lustre: server umount lustre-OST0000 complete [ 2350.561119] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2350.569734] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2356.197111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2356.211993] Lustre: Skipped 4 previous similar messages [ 2356.838991] Lustre: server umount lustre-MDT0000 complete [ 2360.922497] LustreError: 76567:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741705 with bad export cookie 8666334262286203530 [ 2360.932407] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2360.961426] LustreError: 76567:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2361.497713] Lustre: server umount lustre-MDT0001 complete [ 2371.880996] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 00:15:14 (1790741714) [ 2379.032498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2379.626696] LustreError: 80958:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2379.667191] LustreError: 80958:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2379.732213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2384.542672] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2384.868105] LustreError: 80959:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2389.993642] LustreError: 80958:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2395.149506] LustreError: 80959:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2395.851886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2401.557419] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2412.237904] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2419.547457] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2427.369813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2427.803315] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2427.808660] Lustre: Skipped 1 previous similar message [ 2435.017523] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2443.407897] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2448.072696] Lustre: 83631:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting f468ecc7-318e-40a2-81bc-ac1b6e5c0f01 at adminstrative request [ 2449.315898] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2449.368143] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2459.623942] Lustre: MGS: Client f468ecc7-318e-40a2-81bc-ac1b6e5c0f01 (at 0@lo) reconnecting [ 2459.636070] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x78450118685dbdf5 to 0x78450118685dc09c [ 2459.655206] Lustre: MGC192.168.204.136@tcp: Connection restored to 0@lo (at 0@lo) [ 2487.013571] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2487.322340] Lustre: lustre-OST0001: new disk, initializing [ 2487.327975] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2487.334966] Lustre: 84024:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2493.587699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2493.602152] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2493.760612] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2495.332719] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2506.196383] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2514.416754] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2514.426692] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2514.432812] LustreError: Skipped 1 previous similar message [ 2514.448418] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2514.455980] Lustre: Skipped 2 previous similar messages [ 2514.478087] Lustre: Skipped 1 previous similar message [ 2516.279330] Lustre: server umount lustre-OST0001 complete [ 2519.532490] LustreError: 83643:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2519.556203] LustreError: 83643:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2523.106527] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2523.115677] LustreError: Skipped 1 previous similar message [ 2526.911519] Lustre: server umount lustre-OST0000 complete [ 2534.883141] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2534.906771] Lustre: Skipped 3 previous similar messages [ 2539.014376] Lustre: server umount lustre-MDT0000 complete [ 2540.002579] LustreError: 80958:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2540.039060] LustreError: 80958:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2543.820672] LustreError: 80938:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741888 with bad export cookie 8666334262286205084 [ 2543.841355] LustreError: 80938:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2543.842862] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2544.426221] Lustre: server umount lustre-MDT0001 complete [ 2555.319187] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 00:18:18 (1790741898) [ 2557.078383] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2559.317897] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 00:18:21 (1790741901) [ 2561.071502] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2563.071536] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 00:18:26 (1790741906) [ 2564.923133] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2567.183935] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 00:18:29 (1790741909) [ 2569.345886] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2571.578860] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 00:18:34 (1790741914) [ 2573.509617] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2580.994158] loop: module loaded [ 2581.032829] loop0: detected capacity change from 0 to 4294967296 [ 2581.179552] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2583.187188] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2587.393580] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2592.095102] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2622.636725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2622.701295] LustreError: 88151:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2622.708884] LustreError: 88151:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2622.717689] LustreError: 88151:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2622.727377] LustreError: 88151:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2647.207711] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 2660.887787] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2703.575510] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2703.837293] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2703.877900] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2703.961283] Lustre: lustre-MDT0000: new disk, initializing [ 2704.071902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2704.080931] Lustre: Skipped 1 previous similar message [ 2704.099666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2709.310541] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2721.644741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2721.802069] Lustre: 91036:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2722.203147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2727.264737] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2737.637330] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2744.814623] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2751.582798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2751.879956] Lustre: 93008:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2758.219610] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2758.384844] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2759.587785] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2770.386763] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2775.997616] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2776.257390] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2780.162884] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2780.356454] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2784.228401] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2784.231850] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2784.237509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2784.237517] Lustre: Skipped 6 previous similar messages [ 2784.242560] LustreError: Skipped 2 previous similar messages [ 2784.284515] Lustre: Skipped 1 previous similar message [ 2788.816893] Lustre: server umount lustre-OST0000 complete [ 2800.743733] Lustre: server umount lustre-MDT0000 complete [ 2804.708151] LustreError: 91050:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2804.735305] LustreError: 91050:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2805.295549] LustreError: 91030:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742150 with bad export cookie 8666334262286205693 [ 2805.305269] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2805.315891] LustreError: 91030:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2805.766513] Lustre: server umount lustre-MDT0001 complete [ 2815.483683] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 00:22:38 (1790742158) [ 2822.186200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2828.304287] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2837.478146] LustreError: 95421:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2837.517623] LustreError: 95421:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2838.515744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2838.872993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2838.886633] Lustre: Skipped 3 previous similar messages [ 2843.853553] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2853.185102] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2860.490668] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2868.944350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2878.735210] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2889.839473] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2895.427402] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2895.832536] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2901.316618] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2901.567700] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2909.756727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2910.074813] Lustre: lustre-OST0001: new disk, initializing [ 2910.080056] Lustre: Skipped 2 previous similar messages [ 2910.088286] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2910.101944] Lustre: Skipped 3 previous similar messages [ 2910.111633] Lustre: 98792:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2917.947587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2917.954160] Lustre: Skipped 2 previous similar messages [ 2917.958420] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2918.077670] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2918.493892] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2929.801141] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2935.412404] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2935.683554] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2941.017829] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2941.268767] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2961.866300] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2994.786193] Lustre: Failing over lustre-MDT0000 [ 2995.190208] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2995.201554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2995.202563] Lustre: Skipped 4 previous similar messages [ 2995.220044] Lustre: Skipped 8 previous similar messages [ 2996.081170] Lustre: server umount lustre-MDT0000 complete [ 2999.890053] LustreError: 96423:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2999.915086] LustreError: 96423:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3017.696168] Lustre: 66764:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790742345/real 1790742345] req@00000000c4d0a4b7 x1877728363311104/t0(0) o400->MGC192.168.204.136@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790742361 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3017.762243] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3023.552593] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3023.555825] LDISKFS-fs (dm-0): recovery complete [ 3023.562868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3027.975355] LustreError: 66760:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000000aae5865 x1877728363320064/t0(0) o250->MGC192.168.204.136@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3027.999953] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x78450118685e8b17 [ 3028.007249] Lustre: MGC192.168.204.136@tcp: Connection restored to 0@lo (at 0@lo) [ 3028.584778] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3031.574461] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3031.575204] Lustre: 95415:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3033.573842] Lustre: 95420:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3033.597140] Lustre: 95420:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 3033.628170] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3033.797744] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3034.149964] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3034.498918] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3034.509505] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 3034.610550] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3034.827110] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3034.836544] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 3034.849165] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3034.860493] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 3035.522758] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3035.578930] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 3035.930558] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3035.944297] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 3036.924155] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3036.945897] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 5 previous similar messages [ 3037.611478] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3037.640059] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 3037.976164] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3037.998748] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 3041.202626] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3041.233479] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 3041.882523] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3041.919483] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 3042.256166] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3042.265227] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 3049.357557] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3049.383909] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 3050.051226] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3050.071068] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 3050.401389] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3050.414172] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 3065.425329] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3065.455247] Lustre: 101654:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 49 previous similar messages [ 3066.121152] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3066.137560] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3066.464492] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3066.479699] LustreError: 101654:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3091.288168] Lustre: 101654:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3091.306374] Lustre: 101654:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3091.332528] Lustre: 101654:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client e8523677-9ce6-4e96-89a9-8d65db887ef2@192.168.204.36@tcp [ 3091.358191] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3091.368930] Lustre: 101654:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@00000000d1c56391 x1877728345265536/t0(8589934775) o101->e8523677-9ce6-4e96-89a9-8d65db887ef2@192.168.204.36@tcp:226/0 lens 592/0 e 0 to 0 dl 1790742446 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3091.412646] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3091.438747] Lustre: lustre-MDT0000: Denying connection for new client e8523677-9ce6-4e96-89a9-8d65db887ef2 (at 192.168.204.36@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3091.447840] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3091.525966] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3091.536094] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3091.541455] Lustre: Skipped 2 previous similar messages [ 3091.589810] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 3091.593258] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3107.813878] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3111.803804] Lustre: server umount lustre-OST0000 complete [ 3125.732906] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3125.746534] Lustre: Skipped 9 previous similar messages [ 3130.336165] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3130.548402] Lustre: server umount lustre-OST0001 complete [ 3135.972075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3135.999316] Lustre: Skipped 6 previous similar messages [ 3149.280304] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3149.530953] Lustre: server umount lustre-MDT0000 complete [ 3151.329295] LustreError: 96148:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3151.361381] LustreError: 96148:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 47 previous similar messages [ 3153.961651] LustreError: 95401:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742498 with bad export cookie 8666334262286256919 [ 3153.970114] LustreError: 95401:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3153.972401] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3154.017256] LustreError: 66763:0:(import.c:762:ptlrpc_connect_import_locked()) already connecting [ 3154.285910] Lustre: server umount lustre-MDT0001 complete [ 3163.637817] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 00:28:26 (1790742506) [ 3172.669067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3172.679384] Lustre: *** cfs_fail_loc=197, val=0*** [ 3188.323269] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3193.440520] LustreError: 103631:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.136@tcp: failed processing log, type 4: rc = -110 [ 3198.603188] Lustre: *** cfs_fail_loc=197, val=0*** [ 3198.605314] Lustre: Skipped 65 previous similar messages [ 3219.168290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3219.181796] Lustre: Skipped 3 previous similar messages [ 3226.416369] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3234.707795] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3237.122380] Lustre: server umount lustre-OST0000 complete [ 3247.858646] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 00:29:50 (1790742590) [ 3271.008165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3277.221953] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3286.698782] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3290.767929] LustreError: 105285:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3290.784895] LustreError: 105285:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3290.797389] LustreError: 105285:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3291.209886] Lustre: server umount lustre-MDT0000 complete [ 3329.558703] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3341.151896] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3385.111368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3385.340668] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3385.390204] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3385.519599] Lustre: lustre-MDT0000: new disk, initializing [ 3385.637960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3389.392586] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3401.724342] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3401.903222] Lustre: 109459:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3402.360194] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3407.639313] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3419.682307] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3425.189937] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3433.676540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3434.227915] Lustre: 111430:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3440.261965] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3443.497839] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3451.161377] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3456.623225] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3456.837624] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3462.672714] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3462.905896] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3469.286232] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3469.292668] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3469.296474] LustreError: Skipped 4 previous similar messages [ 3469.298874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3469.298883] Lustre: Skipped 6 previous similar messages [ 3469.339192] Lustre: Skipped 1 previous similar message [ 3471.334963] Lustre: server umount lustre-OST0000 complete [ 3482.716928] Lustre: server umount lustre-MDT0000 complete [ 3484.646250] LustreError: 109469:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3484.685723] LustreError: 109469:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3487.156174] LustreError: 109453:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742832 with bad export cookie 8666334262286267545 [ 3487.160425] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3487.166321] LustreError: 109453:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3487.599146] Lustre: server umount lustre-MDT0001 complete [ 3497.615401] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 00:34:00 (1790742840) [ 3507.439668] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 3510.181853] Key type lgssc unregistered [ 3510.466921] LNet: 113915:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3510.475472] LNetError: 113915:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3510.487168] LNet: Removed LNI 192.168.204.136@tcp [ 3511.472186] Key type .llcrypt unregistered [ 3511.474307] Key type ._llcrypt unregistered [ 3530.755927] Key type ._llcrypt registered [ 3530.759724] Key type .llcrypt registered [ 3530.881729] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3548.651782] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3549.908568] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3550.036213] alg: No test for adler32 (adler32-zlib) [ 3551.135227] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 3551.441389] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 3553.168205] Key type lgssc registered [ 3554.329709] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3602.725105] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3602.743139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3603.911611] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3603.935177] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3604.138422] Lustre: lustre-MDT0000: new disk, initializing [ 3604.316885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3604.338780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3609.407577] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3620.992551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3621.176779] Lustre: lustre-OST0000: new disk, initializing [ 3621.179244] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3621.182080] Lustre: Skipped 1 previous similar message [ 3621.185230] Lustre: 118106:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3621.285548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3626.568284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3626.577526] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3626.619733] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3628.886909] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3640.516422] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3640.715397] Lustre: lustre-OST0001: new disk, initializing [ 3640.723960] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3640.759510] Lustre: 118969:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3640.845833] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3647.777408] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3649.617593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3649.629304] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3649.668160] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3659.749518] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3659.761488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3659.769580] Lustre: Skipped 1 previous similar message [ 3663.156216] Lustre: server umount lustre-MDT0000 complete [ 3671.376735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3676.271662] LustreError: 117293:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743021 with bad export cookie 13392159201555696715 [ 3676.272477] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3676.289291] LustreError: 117293:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3676.379675] Lustre: server umount lustre-OST0000 complete [ 3680.622093] Lustre: server umount lustre-OST0001 complete [ 3691.072930] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 00:37:13 (1790743033) [ 3718.734210] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 00:37:41 (1790743061) [ 3730.387971] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3756.597668] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 00:38:19 (1790743099) [ 3800.229231] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3814.774523] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3857.363130] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3857.739733] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3857.780913] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3857.867265] Lustre: lustre-MDT0000: new disk, initializing [ 3857.980550] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3857.999592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3863.441173] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3874.511279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3874.615609] Lustre: 126740:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3874.662778] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3874.668156] Lustre: Skipped 1 previous similar message [ 3874.756243] Lustre: lustre-MDT0001: new disk, initializing [ 3874.865893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3874.905637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3874.916484] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3879.133868] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3891.242873] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3898.516622] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3905.394180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3905.647248] Lustre: lustre-OST0000: new disk, initializing [ 3905.650531] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3905.670659] Lustre: 128710:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3905.768334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3911.282668] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3914.843923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3914.855678] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3914.964171] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3921.344252] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3925.640490] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3925.834226] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3930.153372] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3930.376291] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3935.718521] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3935.724550] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3935.733188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3935.733197] Lustre: Skipped 1 previous similar message [ 3935.747151] Lustre: Skipped 1 previous similar message [ 3935.773990] LustreError: Skipped 1 previous similar message [ 3938.644923] Lustre: server umount lustre-OST0000 complete [ 3945.958484] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3945.969330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3945.976739] Lustre: Skipped 1 previous similar message [ 3945.988379] Lustre: Skipped 2 previous similar messages [ 3948.681567] Lustre: server umount lustre-MDT0000 complete [ 3952.895348] LustreError: 126730:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743297 with bad export cookie 13392159201555697940 [ 3952.918393] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3953.443249] Lustre: server umount lustre-MDT0001 complete [ 3963.698520] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 00:41:46 (1790743306) [ 3971.039899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3971.636956] LustreError: 131123:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3971.665694] LustreError: 131123:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3971.732958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3976.704834] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3977.186894] LustreError: 131124:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3981.294922] LustreError: 131123:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3986.203691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3986.409217] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 3991.015842] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3998.468210] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4003.892564] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4011.521831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4011.902198] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4011.911782] Lustre: Skipped 1 previous similar message [ 4017.307398] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4024.971185] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4038.619235] Lustre: *** cfs_fail_loc=159, val=0*** [ 4038.626060] Lustre: Skipped 1 previous similar message [ 4055.129652] Lustre: lustre-MDT0000: Client 476efc1c-fac9-4f2c-98b7-744312b51692 (at 192.168.204.36@tcp) reconnecting [ 4056.712465] Lustre: *** cfs_fail_loc=159, val=0*** [ 4056.714380] Lustre: Skipped 4 previous similar messages [ 4073.101025] Lustre: lustre-MDT0000: Client 476efc1c-fac9-4f2c-98b7-744312b51692 (at 192.168.204.36@tcp) reconnecting [ 4092.902159] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4092.902899] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4092.932293] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4092.947725] Lustre: Skipped 1 previous similar message [ 4095.982574] Lustre: server umount lustre-OST0000 complete [ 4103.140327] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4103.141320] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4103.162023] Lustre: Skipped 2 previous similar messages [ 4105.920351] Lustre: server umount lustre-MDT0000 complete [ 4108.257650] LustreError: 131124:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4108.292050] LustreError: 131124:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4109.459222] LustreError: 131104:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743454 with bad export cookie 13392159201555698822 [ 4109.463735] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4109.475036] LustreError: 131104:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 4109.750417] Lustre: server umount lustre-MDT0001 complete [ 4121.364200] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4124.136433] Key type lgssc unregistered [ 4124.434911] LNet: 134924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4124.440593] LNetError: 134924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4124.463095] LNet: Removed LNI 192.168.204.136@tcp [ 4125.281162] Key type .llcrypt unregistered [ 4125.283735] Key type ._llcrypt unregistered [ 4150.300634] Key type ._llcrypt registered [ 4150.303755] Key type .llcrypt registered [ 4150.434762] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4151.392859] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4151.443106] alg: No test for adler32 (adler32-zlib) [ 4152.513603] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 4152.792706] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4154.528328] Key type lgssc registered [ 4155.784790] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4164.000457] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4164.029614] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4165.653317] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4170.043543] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4175.847253] LustreError: 136356:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4175.870526] LustreError: 136356:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4178.235465] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4178.631258] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4182.609924] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4189.502181] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4195.045966] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4204.990029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4205.588830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4210.670352] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4210.914957] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4217.497562] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4229.714217] Lustre: *** cfs_fail_loc=159, val=0*** [ 4246.082801] Lustre: lustre-MDT0000: Client e0da44ec-e3a7-409d-9bba-0015d522a32d (at 192.168.204.36@tcp) reconnecting [ 4248.661454] Lustre: *** cfs_fail_loc=159, val=0*** [ 4248.666992] Lustre: Skipped 1 previous similar message [ 4265.046034] Lustre: lustre-MDT0001: Client e0da44ec-e3a7-409d-9bba-0015d522a32d (at 192.168.204.36@tcp) reconnecting [ 4266.766317] Lustre: *** cfs_fail_loc=159, val=0*** [ 4282.944059] Lustre: lustre-MDT0001: Client e0da44ec-e3a7-409d-9bba-0015d522a32d (at 192.168.204.36@tcp) reconnecting [ 4299.330926] Lustre: 139417:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4302.610359] Lustre: *** cfs_fail_loc=159, val=0*** [ 4302.613153] Lustre: Skipped 4 previous similar messages [ 4318.803450] Lustre: lustre-MDT0000: Client 7e2364b4-aa88-4270-b350-675d4d80eca7 (at 192.168.204.36@tcp) reconnecting [ 4321.068951] Lustre: *** cfs_fail_loc=159, val=0*** [ 4321.074764] Lustre: Skipped 14 previous similar messages [ 4336.781608] Lustre: lustre-MDT0000: Client 7e2364b4-aa88-4270-b350-675d4d80eca7 (at 192.168.204.36@tcp) reconnecting [ 4338.493312] Lustre: 139708:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4357.097082] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4357.098030] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4357.111916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4357.132463] Lustre: Skipped 1 previous similar message [ 4362.213297] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4362.223036] Lustre: Skipped 1 previous similar message [ 4367.330541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4367.341465] Lustre: Skipped 1 previous similar message [ 4369.376539] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4369.678591] Lustre: server umount lustre-OST0000 complete [ 4377.570740] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.583163] Lustre: Skipped 1 previous similar message [ 4377.587776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4379.948803] Lustre: server umount lustre-MDT0000 complete [ 4382.692256] LustreError: 136356:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4382.712387] LustreError: 136356:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4383.973182] LustreError: 136337:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743728 with bad export cookie 3003229384953576046 [ 4383.975407] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4383.985071] LustreError: 136337:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4384.328594] Lustre: server umount lustre-MDT0001 complete [ 4395.464205] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4398.430987] Key type lgssc unregistered [ 4398.748283] LNet: 140787:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4398.755258] LNetError: 140787:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4398.772510] LNet: Removed LNI 192.168.204.136@tcp [ 4399.589095] Key type .llcrypt unregistered [ 4399.592102] Key type ._llcrypt unregistered [ 4428.106879] Key type ._llcrypt registered [ 4428.108942] Key type .llcrypt registered [ 4428.220940] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4429.323306] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4429.343257] alg: No test for adler32 (adler32-zlib) [ 4430.522151] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 4430.815939] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4432.496575] Key type lgssc registered [ 4433.878472] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4441.454892] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4441.482386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4442.921354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4447.934488] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4452.326617] LustreError: 142219:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4452.345643] LustreError: 142219:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4456.554535] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4456.834619] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4460.903236] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4469.363889] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4474.152788] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4480.427446] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4480.753731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4481.787129] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4486.864895] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4487.952205] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4494.923891] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4500.525445] Lustre: 144801:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4503.367269] Lustre: 144850:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4505.169838] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4506.100337] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4510.017032] Lustre: 144997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4510.842590] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4511.687415] Lustre: 145095:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4517.858244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4517.864856] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4517.874461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4518.881904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4518.883048] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4518.901810] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4522.167154] Lustre: server umount lustre-OST0000 complete [ 4528.612761] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4528.623284] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4528.639093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4528.646152] Lustre: Skipped 1 previous similar message [ 4534.243027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4534.249263] Lustre: Skipped 3 previous similar messages [ 4539.363195] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4539.367875] Lustre: Skipped 1 previous similar message [ 4540.384213] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4540.799843] Lustre: server umount lustre-MDT0000 complete [ 4543.459097] LustreError: 142218:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4543.489117] LustreError: 142218:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4545.038506] LustreError: 142199:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743889 with bad export cookie 3838772513014427946 [ 4545.054103] LustreError: 142199:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4545.059150] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4545.470401] Lustre: server umount lustre-MDT0001 complete [ 4556.933626] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4559.628724] Key type lgssc unregistered [ 4559.870893] LNet: 146156:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4559.875932] LNetError: 146156:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4559.901242] LNet: Removed LNI 192.168.204.136@tcp [ 4560.830098] Key type .llcrypt unregistered [ 4560.832385] Key type ._llcrypt unregistered [ 4587.370633] Key type ._llcrypt registered [ 4587.375361] Key type .llcrypt registered [ 4587.513076] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4588.577570] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4588.635481] alg: No test for adler32 (adler32-zlib) [ 4589.829485] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 4590.098840] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4591.744238] Key type lgssc registered [ 4592.832291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4600.257847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4600.309047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4601.899936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4606.664154] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4612.070905] LustreError: 147587:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4612.095479] LustreError: 147587:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4614.827040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4615.348689] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4619.802251] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4627.764427] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4633.708509] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4640.979458] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4641.298228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4646.400515] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4647.580634] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4655.017335] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4660.734083] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4663.904961] Lustre: *** cfs_fail_loc=159, val=0*** [ 4680.276759] Lustre: lustre-MDT0000: Client 6db6638b-05ff-4ce1-98c2-dfd8c81e1401 (at 192.168.204.36@tcp) reconnecting [ 4697.576092] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4697.596638] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4697.626992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4701.666724] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4701.667776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4701.682781] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4703.404422] Lustre: server umount lustre-OST0000 complete [ 4707.299563] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4707.315198] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4707.325335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4707.331706] Lustre: Skipped 1 previous similar message [ 4711.910128] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4711.910876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4711.935918] Lustre: Skipped 1 previous similar message [ 4713.360082] Lustre: server umount lustre-MDT0000 complete [ 4717.034756] LustreError: 147581:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4717.051048] LustreError: 147581:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4717.381636] LustreError: 150199:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744062 with bad export cookie 2372815052240175693 [ 4717.384978] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4717.392861] LustreError: 150199:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4717.790599] Lustre: server umount lustre-MDT0001 complete [ 4727.641532] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4729.974881] Key type lgssc unregistered [ 4730.187839] LNet: 151252:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4730.197243] LNetError: 151252:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4730.210491] LNet: Removed LNI 192.168.204.136@tcp [ 4730.923182] Key type .llcrypt unregistered [ 4730.925559] Key type ._llcrypt unregistered [ 4754.096664] Key type ._llcrypt registered [ 4754.099127] Key type .llcrypt registered [ 4754.202942] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4754.970280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4755.055389] alg: No test for adler32 (adler32-zlib) [ 4756.053977] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 4756.296587] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4757.969072] Key type lgssc registered [ 4759.200888] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4766.399129] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4766.424596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4767.851811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4772.070325] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4777.446404] LustreError: 152683:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4777.471922] LustreError: 152683:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4779.637944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4780.091656] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4784.355185] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4792.143782] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4797.134390] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4803.341866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4803.702258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4804.728087] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4809.684228] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4812.920724] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4817.552724] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4836.865026] Lustre: lustre-MDT0001: evicting 000a9a80-7fb3-447d-a45c-5da9de18cf16 (at 192.168.204.36@tcp) by administrative request [ 4836.870891] Lustre: Skipped 1 previous similar message [ 4866.529955] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4866.541891] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4866.554931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4867.554475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4867.556192] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4870.688828] Lustre: server umount lustre-OST0000 complete [ 4877.283043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4877.302363] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4877.327324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4877.335715] Lustre: Skipped 1 previous similar message [ 4880.492819] Lustre: server umount lustre-MDT0000 complete [ 4882.915433] LustreError: 152683:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4882.936461] LustreError: 152683:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4884.244898] LustreError: 152663:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744229 with bad export cookie 16002458407225602834 [ 4884.262891] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4884.578536] Lustre: server umount lustre-MDT0001 complete [ 4895.231347] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4898.609426] Key type lgssc unregistered [ 4898.919870] LNet: 156751:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4898.928967] LNetError: 156751:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4900.015685] LNet: Removed LNI 192.168.204.136@tcp [ 4900.960921] Key type .llcrypt unregistered [ 4900.967250] Key type ._llcrypt unregistered [ 4983.955662] Key type ._llcrypt registered [ 4983.966372] Key type .llcrypt registered [ 4984.063797] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4984.684804] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4984.898561] alg: No test for adler32 (adler32-zlib) [ 4985.916961] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 4986.142716] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4987.872320] Key type lgssc registered [ 4988.998497] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4997.247631] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4997.281041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4998.932149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5004.583583] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5008.354556] LustreError: 159257:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5008.379943] LustreError: 159257:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5013.494063] LustreError: 159258:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5015.001551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5015.388352] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5020.480731] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5028.847199] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5035.282415] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5042.797482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5043.153588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5048.325660] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5050.750383] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5058.993825] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5060.600339] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5068.770288] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5068.784155] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5068.806129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5070.820047] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5070.831132] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5070.832206] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5070.868207] Lustre: Skipped 1 previous similar message [ 5073.976276] Lustre: server umount lustre-OST0000 complete [ 5081.058530] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5081.062933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5081.073742] Lustre: Skipped 1 previous similar message [ 5081.084483] Lustre: Skipped 1 previous similar message [ 5084.016693] Lustre: server umount lustre-MDT0000 complete [ 5086.184602] LustreError: 159258:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5086.203656] LustreError: 159258:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5087.673384] LustreError: 159240:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744432 with bad export cookie 4903633091949739433 [ 5087.681791] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5087.685026] LustreError: 159240:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5088.000873] Lustre: server umount lustre-MDT0001 complete [ 5100.351178] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 5103.288625] Key type lgssc unregistered [ 5103.630181] LNet: 162800:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5103.644497] LNetError: 162800:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5103.668115] LNet: Removed LNI 192.168.204.136@tcp [ 5104.742621] Key type .llcrypt unregistered [ 5104.746051] Key type ._llcrypt unregistered [ 5131.296979] Key type ._llcrypt registered [ 5131.299822] Key type .llcrypt registered [ 5131.452690] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 5132.546427] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5132.574339] alg: No test for adler32 (adler32-zlib) [ 5134.070226] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 5134.482740] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 5136.240409] Key type lgssc registered [ 5137.936074] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5156.659595] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 01:01:39 (1790744499) [ 5175.782617] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 5185.912304] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 5228.104727] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5228.138361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5229.393989] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5229.429648] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5229.550268] Lustre: lustre-MDT0000: new disk, initializing [ 5229.656378] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5229.676498] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5234.765503] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5246.740168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5246.901483] Lustre: 167093:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5247.039039] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5247.047187] Lustre: Skipped 1 previous similar message [ 5247.192380] Lustre: lustre-MDT0001: new disk, initializing [ 5247.362836] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5247.426285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5247.444888] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5252.520389] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5263.045210] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5269.310742] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5276.633264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5277.007795] Lustre: lustre-OST0000: new disk, initializing [ 5277.013275] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5277.023534] Lustre: 169060:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5277.185262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5284.000359] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5284.991067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5285.013749] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5285.153422] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5294.496497] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5305.829236] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5305.840334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5305.849403] LustreError: Skipped 1 previous similar message [ 5305.860204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5305.870996] Lustre: Skipped 1 previous similar message [ 5305.899771] Lustre: Skipped 1 previous similar message [ 5310.674456] Lustre: server umount lustre-OST0000 complete [ 5316.067416] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5316.105730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5316.126248] Lustre: Skipped 1 previous similar message [ 5321.187832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5321.194990] Lustre: Skipped 1 previous similar message [ 5322.116836] Lustre: server umount lustre-MDT0000 complete [ 5327.137476] LustreError: 167085:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744672 with bad export cookie 14847003758356137563 [ 5327.147174] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5327.661996] Lustre: server umount lustre-MDT0001 complete [ 5343.317722] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 5347.554251] Key type lgssc unregistered [ 5348.002620] LNet: 170966:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5348.012978] LNetError: 170966:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5348.045588] LNet: Removed LNI 192.168.204.136@tcp [ 5349.258717] Key type .llcrypt unregistered [ 5349.261656] Key type ._llcrypt unregistered [ 5379.307689] Key type ._llcrypt registered [ 5379.310108] Key type .llcrypt registered [ 5379.390732] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 5380.468028] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5380.512086] alg: No test for adler32 (adler32-zlib) [ 5381.625632] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 5381.946687] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 5383.688229] Key type lgssc registered [ 5385.066955] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5393.093872] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5393.104240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5394.637072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5400.475469] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5404.138212] LustreError: 172399:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5404.165096] LustreError: 172399:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5409.253426] LustreError: 172398:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5410.637561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5411.283188] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5415.506828] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5422.199082] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5427.008234] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5434.183678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5434.518137] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5439.682166] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5441.099177] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 5450.436352] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5458.268687] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5458.307530] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5458.330207] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5458.346398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.375405] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5459.671696] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5459.705098] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5459.733305] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5459.747305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.776522] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5460.924078] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5460.953547] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5460.967804] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5460.977235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.991773] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5461.980062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.963727] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5462.968554] Lustre: Skipped 1 previous similar message [ 5462.994719] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5463.011294] Lustre: Skipped 1 previous similar message [ 5463.018592] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5463.031899] Lustre: Skipped 1 previous similar message [ 5463.041542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.058386] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5463.064613] Lustre: Skipped 1 previous similar message [ 5463.936871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.937577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.904489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.819761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.565419] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5467.572437] Lustre: Skipped 4 previous similar messages [ 5467.583672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5467.597990] Lustre: Skipped 4 previous similar messages [ 5467.606233] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5467.619265] Lustre: Skipped 4 previous similar messages [ 5467.624686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.635495] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5467.640534] Lustre: Skipped 4 previous similar messages [ 5468.910570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.720894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.537065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.424409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.613103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.708674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.578978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.441682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.565901] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5476.571515] Lustre: Skipped 8 previous similar messages [ 5476.605685] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5476.630353] Lustre: Skipped 8 previous similar messages [ 5476.646876] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5476.658185] Lustre: Skipped 8 previous similar messages [ 5476.679951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.690058] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5476.699034] Lustre: Skipped 8 previous similar messages [ 5477.718033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.955559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.157983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.117452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.437478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.424416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.781527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.859641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.621894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.488144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.660371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.055199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.197963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.370496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.423285] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5493.431033] Lustre: Skipped 14 previous similar messages [ 5493.449738] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5493.463246] Lustre: Skipped 14 previous similar messages [ 5493.468597] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5493.473848] Lustre: Skipped 14 previous similar messages [ 5493.477598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.488017] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5493.501354] Lustre: Skipped 14 previous similar messages [ 5494.465872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.669165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.661370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.509718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.374543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.379611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.277246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.200654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.150161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.105393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.243440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.155667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.143525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.317140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.508125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.692290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.657160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.523706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.338889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.402640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.308218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.234304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.145080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.983627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.005306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.900447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.752681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.800558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.170268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.368249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.540039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.416836] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5526.446820] Lustre: Skipped 31 previous similar messages [ 5526.497337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5526.510502] Lustre: Skipped 31 previous similar messages [ 5526.516470] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5526.524873] Lustre: Skipped 31 previous similar messages [ 5526.528577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.542081] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5526.561076] Lustre: Skipped 31 previous similar messages [ 5528.255829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.861395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.470084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.995718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.445723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.207177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.370413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.452888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.890068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.873454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.796187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.917323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.821454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.976149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.089953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.087588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.048864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.120962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.057407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.524169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.948189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.411300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.737788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.225265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.549683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.688977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.361164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.615402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.635541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.656929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.657766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.641342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.560615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.422472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.402700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.244051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.181089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.062046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.046799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.977375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.014734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.866602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.834602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.049383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.051028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.106219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.015756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.000146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.906655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.877046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.767922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.609499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.355098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.275266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.244980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.411314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.313758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.352410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.339771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.186162] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5591.191228] Lustre: Skipped 59 previous similar messages [ 5591.206988] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5591.218971] Lustre: Skipped 59 previous similar messages [ 5591.222929] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5591.230186] Lustre: Skipped 59 previous similar messages [ 5591.234840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.249252] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5591.260452] Lustre: Skipped 59 previous similar messages [ 5592.020878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.860468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.753917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.692807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.725634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.799843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.874589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.029373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.049644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.830948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.718472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.423125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.313526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.172320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.089698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.042753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.968605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.976591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.070744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.995775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.900377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.793758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.806989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.807742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.897705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.878664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.086706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.196060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.331687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.356871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.464417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.486526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.322250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.422589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.391477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.478153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.841153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.981640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.333111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.350673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.377841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.392176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.668396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.005345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.998833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.912779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.958891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.918298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.105654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.080269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.026905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.004980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.874355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.744173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.752577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.637120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.552903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.471989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.597169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.512682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.414794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.275728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.249963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.130869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.123673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.971574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.852591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.827708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.801640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.735826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.741838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.527422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.564702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.471202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.520882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.505354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.488827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.393891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.247329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.281303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.392304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.543341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.661835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.699194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.678632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.720942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.644275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.724271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.694135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.684743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.576067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.606093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.537426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.391371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.527540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.524635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.498823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.521873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.435902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.371632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.299151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.128172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.061418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.039860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.098476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.020649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.835646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.615718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.429168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.223465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.170340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.198622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.204797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.242758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.433578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.659903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.584174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.625461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.651819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.716792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.784053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.808604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.658862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.718489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.942354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.062906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.329635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.512542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.462777] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5719.469502] Lustre: Skipped 128 previous similar messages [ 5719.496547] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5719.513317] Lustre: Skipped 128 previous similar messages [ 5719.520537] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5719.527441] Lustre: Skipped 128 previous similar messages [ 5719.535378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.558495] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5719.566875] Lustre: Skipped 128 previous similar messages [ 5720.644912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.537281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.483405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.343398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.211862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.262099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.346683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.564728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.673041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.722081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.909270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.026367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.999292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.874207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.737902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.649198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.488584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.398883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.097702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.757945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.599869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.397100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.152948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.089295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.927951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.872690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.604289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.349637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.062842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.906852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.827899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.564768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.518955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.577901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.458622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.800565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.737508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.939471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.249860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.740451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.070871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.566563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.771373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.900345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.286778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.449621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.728864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.325267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.228462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.010181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.738501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.543182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.504564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.457416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.239315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.232693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.017169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.104427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.988489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.809232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.619506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.662049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.468970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.150939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.971234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.973278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.823673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.650980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.337600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.143000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.020988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.970645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.880119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.780424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.802096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.806600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.671624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.545669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.435857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.334695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.213558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.293578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.381359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.182788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.290524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.232299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.167516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.921702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.850914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.706864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.774449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.768145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.745845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.942200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.986225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.049868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.112827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.231271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.569226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.951563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.211072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.763615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.842808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.494247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.401764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.204036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.470677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.466362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.848109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.233331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.427535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.861446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.353569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.675918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.250272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.927084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.840792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.278946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.247416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.736158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.089776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.887754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.948342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.380837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.603693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.529467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.879220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.977668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.369523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.631857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.780958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.655233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.968881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.759621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.683148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.582743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.467872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.252047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.237907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.138938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.072328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.994778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.953176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.024330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.175467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.218331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.211082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.209554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.207491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.298944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.419437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.528730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.538193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.528313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.670085] LustreError: 172418:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5879.681885] LustreError: 172418:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5879.692519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.833436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.788429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.682640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.621741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.800566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.734964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.805684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.925256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.826543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.753801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.880726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.935719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.985488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.899410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.899876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.680322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.600712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.371192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.386059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.311769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.339086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.257369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.196571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.121066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.984397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.913683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.901374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.768575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.659773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.426650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.393435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.362339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.457704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.506409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.357995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.424961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.368260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.438882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.320419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.186916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.129173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.958249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.772821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.612354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.371617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.136489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.951535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.006673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.042489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.936435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.786477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.639799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.552322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.359870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.172581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.054638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.868868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.822589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.957418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.867019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.841907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.896409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.884515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.804801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.824324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.776578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.703201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.774708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.948332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.129781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.081816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.137338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.181036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.400892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.427982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.316239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.421737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.368633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.332472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.420681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.435646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.588425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.589192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.695812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.715180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.603280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.457900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.394677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.277330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.324606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.172104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.222896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.162429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.207701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.031775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.949850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.873614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.804958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.788783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.840550] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5975.854486] Lustre: Skipped 254 previous similar messages [ 5975.875465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5975.893211] Lustre: Skipped 254 previous similar messages [ 5975.903474] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5975.911095] Lustre: Skipped 254 previous similar messages [ 5975.915724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.935797] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5975.939451] Lustre: Skipped 254 previous similar messages [ 5977.063485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.997164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.910732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.964511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.936862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.040545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.945205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.062388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.333696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.372902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.463908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.556801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.462910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.319545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.080105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.891338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.730744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.775945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.661412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.830959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.924574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.950062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.037832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.885602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.919929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.950330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.112193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.219356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.248312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.316094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.358851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.523591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.561950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.695140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.661408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.776510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.893297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.886372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.966271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.995210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.118525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.076895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.133971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.328972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.248066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.367977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.307878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.212658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.193681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.076356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.078406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.983581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.904418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.930052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.928831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.104810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.182608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.308261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.272993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.387896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.462364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.454797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.479461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.539218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.633682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.727286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.700754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.773985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.720221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.709342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.661144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.746324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.020746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.932566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.988705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.179191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.229476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.166596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.140595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.994327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.927053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.902057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.908808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.754417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.783517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.814850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.043947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.108413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.177402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.238237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.239710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.274907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.486432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.357685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.284385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.907527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.713451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.345885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.075789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.961217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.166191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.131905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.227808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.364794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.197893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.207884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.300264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.560038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.693598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.648422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.647950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.798550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.969796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.152592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.100809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.066657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.074052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.009261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.965729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.884637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.768867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.783387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.720334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.840151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.050473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.019141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.120403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.142193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.256182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.148981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.176040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.195875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.210945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.224805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.314579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.170559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.144265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.348559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.362926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.489559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.505680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.516126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.628989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.930610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.964627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.976916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.942396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.968918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.026344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.206578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.210697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.444670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.398382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.336594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.387226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.326264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.333744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.247325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.113946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.968365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.083840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.003559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.957968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.778749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.882065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.646147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.570506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.429281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.382401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.464816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.500625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.678301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.623428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.452835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.253340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.251355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.164311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.031622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.805061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.671379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.487884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.362980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.176993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.020127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.845230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.725205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.559158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.473292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.276389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.260934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.189707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.232880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.122796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.136991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.103816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.005672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.076077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.065339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.111548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.258803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.457701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.847688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.038165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.996830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.104975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.045642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.172706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.016044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.962941] LustreError: 172418:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6185.973493] LustreError: 172418:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6185.974840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.761444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.953418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.999575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.004726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.042539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.023827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.049745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.313962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.303883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.536954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.614992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.706974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.633024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.820930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.071534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.194667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.254228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.557838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.671561] LustreError: 172418:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6206.687364] LustreError: 172418:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6206.710852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.878481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.094035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.319436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.425053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.531476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.554178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.577693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.472885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.710870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.745397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.903742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.817124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.915444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.916080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.832587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.701772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.681327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.518919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.502619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.252613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.163728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.996913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.825864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.696567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.606568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.516692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.372542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.264259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.171436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.014707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.935758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.996858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.053676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.148229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.075774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.156147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.229544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.467635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.644308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.761131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.945645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.016133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.103751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.183447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.236667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.375036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.337690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.206655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.262356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.161878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.205196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.226166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.278175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.151576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.176311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.282409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.375177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.283979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.204586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.143856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.195762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.058798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.118098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.936060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.996756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.907994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.999907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.003783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.935603] LustreError: 172418:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6275.943960] LustreError: 172418:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6275.944575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.804107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.960412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.947457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.966400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.950632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.963638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.845057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.946967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.837217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.094269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.102529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.004522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.918836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.016144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.995862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.030943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.935250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.034302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.083539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.933551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.925634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.754333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.506721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.282756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.183887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.926939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.607119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.307141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.044225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.657062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.409219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.064305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.716220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.403293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.242302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.057593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.883721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.805312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.891629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.080764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.991759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.051493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.203265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.454969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.660566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.823156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.910482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.081551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.273160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.386250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.464793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.665519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.592349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.407277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.752319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.923079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.854400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.019034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.946078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.991389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.189428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.177601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.248500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.482192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.564487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.532547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.217364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.016571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.081765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.051255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.233824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.372561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.382657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.327983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.321491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.448982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.545047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.873290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.094208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.218298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.370076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.602371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.573054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.614631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.606727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.610068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.565721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.545291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.567980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.558810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.523345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.756202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.778303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.828767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.828121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.770161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.830803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.747698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.555498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.505394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.509544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.457520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.392108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.466752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.420945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.356895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.230847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.138083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.091410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.047471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.918470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.998129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.882838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.809150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.747393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.687887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.605417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.430499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.296783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.192503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.095143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.867721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.593956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.261612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.982505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.799458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.799875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.637566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.475417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.385980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.198861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.117932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.069822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.116731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.128221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.106452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.114834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.943313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.906535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.728673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.641750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.520563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.405179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.338787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.305935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.276676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.240581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.170804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.978185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.925051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.888870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.880995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.823674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.982733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.833417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.770907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.543449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.445816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.774472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.006027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.290351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.292209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.670095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.351632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.679579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.887266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.264847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.531420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.785273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.332735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.186686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.620299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.676318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.681542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.608334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.840244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.622904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.475098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.566910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.762522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.858918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.049210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.988061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.009912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.825638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.602108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.429226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.503443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.266032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.196363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.050809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.833827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.631232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6919.588530] LustreError: 174218:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6919.596526] LustreError: 172418:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7635.439506] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7635.441995] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7635.465679] Lustre: Skipped 491 previous similar messages [ 7635.484073] Lustre: Skipped 1 previous similar message [ 7640.561710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7640.571323] Lustre: Skipped 1 previous similar message [ 7641.333297] Lustre: server umount lustre-OST0000 complete [ 7645.673856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7645.683876] Lustre: Skipped 1 previous similar message [ 7650.786734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7650.801248] Lustre: Skipped 1 previous similar message [ 7651.748876] Lustre: server umount lustre-MDT0000 complete [ 7655.914396] LustreError: 226233:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7655.936556] LustreError: 226233:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7656.061111] LustreError: 172379:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747000 with bad export cookie 2446760591740085677 [ 7656.063336] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7656.083282] LustreError: 172379:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7656.526905] Lustre: server umount lustre-MDT0001 complete [ 7667.256826] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 7670.140220] Key type lgssc unregistered [ 7670.444547] LNet: 227678:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7670.455420] LNetError: 227678:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7670.475752] LNet: Removed LNI 192.168.204.136@tcp [ 7671.212107] Key type .llcrypt unregistered [ 7671.216567] Key type ._llcrypt unregistered [ 7695.248408] Key type ._llcrypt registered [ 7695.268753] Key type .llcrypt registered [ 7695.485426] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 7696.797250] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7696.830205] alg: No test for adler32 (adler32-zlib) [ 7697.877847] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 7698.080331] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 7699.801102] Key type lgssc registered [ 7701.072579] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7708.981312] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7709.008916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7710.509842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7715.359727] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7720.934774] LustreError: 229110:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7720.958937] LustreError: 229110:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7723.601598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7724.101186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7729.237379] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7738.278170] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7744.119185] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7750.682593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7750.999301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7755.141828] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7757.008814] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7763.489351] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7771.408197] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7773.170884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7773.185506] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7773.200401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7773.222369] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7773.225665] LustreError: 229130:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7775.296701] Lustre: lustre-OST0000: Not available for connect from 192.168.204.36@tcp (stopping) [ 7778.793304] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7778.800771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7778.804622] Lustre: Skipped 1 previous similar message [ 7778.822032] Lustre: Skipped 1 previous similar message [ 7780.374616] Lustre: lustre-OST0000: Not available for connect from 192.168.204.36@tcp (stopping) [ 7781.306299] Lustre: server umount lustre-OST0000 complete [ 7793.289258] LustreError: 229130:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7816.024526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7816.421902] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7818.484545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.519046] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.520368] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7824.128344] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7833.563841] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7835.980566] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7841.817914] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff990e5201a800.ost_server_uuid 50 [ 7843.773781] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff990e5201a800.ost_server_uuid in FULL state after 0 sec [ 7849.720031] Lustre: server umount lustre-OST0000 complete [ 7854.564265] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7854.570122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7854.594716] Lustre: Skipped 1 previous similar message [ 7854.610045] Lustre: Skipped 1 previous similar message [ 7859.684493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7859.689114] Lustre: Skipped 1 previous similar message [ 7863.784244] LustreError: 229130:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000ca17fae6 x1877734370372736/t0(0) o7->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 264/240 e 0 to 0 dl 0 ref 2 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7868.384137] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7868.765954] Lustre: server umount lustre-MDT0000 complete [ 7869.926832] LustreError: 229110:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7869.964470] LustreError: 229110:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7870.950337] Lustre: 228410:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790747199/real 1790747199] req@0000000007e75bbc x1877734370371584/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1790747215 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 7870.957253] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7870.987672] Lustre: 228410:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 7871.054105] LustreError: 229111:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7871.097821] LustreError: 229111:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7872.907342] LustreError: 233571:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747217 with bad export cookie 3309436507339849790 [ 7872.916837] LustreError: 233571:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7872.917481] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7873.436842] Lustre: server umount lustre-MDT0001 complete [ 7887.864255] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 7891.507766] Key type lgssc unregistered [ 7891.892878] LNet: 234323:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7891.898853] LNetError: 234323:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7891.913982] LNet: Removed LNI 192.168.204.136@tcp [ 7893.074257] Key type .llcrypt unregistered [ 7893.076379] Key type ._llcrypt unregistered [ 7931.890648] Key type ._llcrypt registered [ 7931.893750] Key type .llcrypt registered [ 7932.054676] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 7933.157866] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7933.366089] alg: No test for adler32 (adler32-zlib) [ 7934.573154] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 7935.003106] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 7936.704283] Key type lgssc registered [ 7938.396812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7949.112631] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7949.146754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7950.887035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7956.320568] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7961.060078] LustreError: 236071:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7961.072302] LustreError: 236071:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7964.888638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7965.225836] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7970.416853] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7973.207517] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7979.921289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7980.492198] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7987.459354] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 7995.917026] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7998.028500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7998.450851] Lustre: lustre-OST0001: new disk, initializing [ 7998.467419] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7998.482134] Lustre: 238380:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7998.694623] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8004.262614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8004.282143] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8004.388973] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8006.217750] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8018.730565] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8030.263911] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8045.543335] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8045.553535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8045.560478] Lustre: Skipped 3 previous similar messages [ 8045.576729] Lustre: Skipped 3 previous similar messages [ 8050.679834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8050.686299] Lustre: Skipped 2 previous similar messages [ 8055.784947] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8055.796619] Lustre: Skipped 3 previous similar messages [ 8059.360338] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8059.907975] Lustre: server umount lustre-MDT0000 complete [ 8060.902476] LustreError: 236072:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8060.930041] LustreError: 236072:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8065.044514] LustreError: 237205:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747409 with bad export cookie 12204917029467767946 [ 8065.063613] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8065.076488] LustreError: 237205:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8065.589796] Lustre: server umount lustre-MDT0001 complete [ 8080.449671] Lustre: server umount lustre-OST0000 complete [ 8081.184552] Lustre: 235323:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790747410/real 1790747410] req@ffffa0def0de5880 x1877734618926208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790747426 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8081.239804] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8085.134547] Lustre: server umount lustre-OST0001 complete [ 8109.396387] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 8120.951352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8121.470034] LustreError: 241492:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8121.613223] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8126.675252] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8126.946525] LustreError: 241493:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8132.069193] LustreError: 241492:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8135.593598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8135.960315] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8140.281301] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8143.718988] Lustre: 242636:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8151.353154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8158.213927] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8160.944852] LustreError: 242990:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8160.959303] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8166.850983] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8167.155131] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8167.160348] Lustre: Skipped 1 previous similar message [ 8168.181404] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8168.184808] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8173.285550] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8181.420823] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8184.752138] Lustre: 244507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8197.607141] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8197.610265] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8197.624462] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8201.895044] Lustre: server umount mylustre-MDT0000 complete [ 8204.268267] LustreError: 241493:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8204.284145] LustreError: 241493:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8205.985417] LustreError: 241467:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747550 with bad export cookie 12204917029467770809 [ 8206.004081] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8206.299405] Lustre: server umount mylustre-MDT0001 complete [ 8220.555838] Lustre: server umount mylustre-OST0000 complete [ 8235.484633] Lustre: server umount mylustre-OST0001 complete [ 8263.968532] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 8276.823985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8277.367922] LustreError: 246821:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8277.460352] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8283.041852] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8291.623802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8295.789188] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8298.315950] Lustre: 247962:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8304.428527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8310.370360] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8316.157520] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 8318.130819] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8322.437350] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8322.446220] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8325.079045] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8333.940589] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8337.783371] Lustre: 249837:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8348.129466] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8348.137652] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8348.141120] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8348.141127] Lustre: Skipped 4 previous similar messages [ 8348.174721] Lustre: Skipped 5 previous similar messages [ 8351.132602] Lustre: server umount tfs-MDT0000 complete [ 8353.250773] LustreError: 247554:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8353.280748] LustreError: 247554:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 8354.720480] LustreError: 246798:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747699 with bad export cookie 12204917029467772979 [ 8354.727969] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8354.738686] LustreError: 246798:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 8355.237590] Lustre: server umount tfs-MDT0001 complete [ 8369.763069] Lustre: server umount tfs-OST0000 complete [ 8374.752418] Lustre: 235325:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790747703/real 1790747703] req@ffffa0def8ec1180 x1877734619039872/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790747719 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8374.774247] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8374.783928] Lustre: Skipped 1 previous similar message [ 8375.038723] Lustre: server umount tfs-OST0001 complete [ 8400.370383] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 8411.786072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8412.339376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8412.347417] Lustre: Skipped 3 previous similar messages [ 8417.208646] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8427.254707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8433.124204] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8436.304672] Lustre: 253231:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8443.810137] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8450.707962] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8452.556121] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 8457.743382] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8460.355075] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8465.917773] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8467.676794] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8475.305314] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8479.249938] Lustre: 255103:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8487.333816] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 01:57:09 (1790747829) [ 8504.293059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8504.305988] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8504.336107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8504.348372] Lustre: Skipped 3 previous similar messages [ 8510.777538] Lustre: server umount lustre-MDT0000 complete [ 8511.980370] LustreError: 254968:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8511.999599] LustreError: 254968:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 8515.051511] LustreError: 252066:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747859 with bad export cookie 12204917029467774904 [ 8515.057379] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8515.061138] LustreError: 252066:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8524.194795] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8524.201368] Lustre: Skipped 9 previous similar messages [ 8529.377633] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8529.729673] Lustre: server umount lustre-MDT0001 complete [ 8539.298662] Lustre: server umount lustre-OST0000 complete [ 8549.524516] Lustre: server umount lustre-OST0001 complete [ 8556.921030] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 8568.601664] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 8607.625713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8607.859893] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8607.884437] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8607.975097] Lustre: lustre-MDT0000: new disk, initializing [ 8608.109512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8608.117282] Lustre: Skipped 3 previous similar messages [ 8608.133463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8613.223802] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8625.510818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8625.735678] Lustre: 258288:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8625.862110] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8625.867620] Lustre: Skipped 1 previous similar message [ 8626.139086] Lustre: lustre-MDT0001: new disk, initializing [ 8626.354063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8626.365205] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8632.835957] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8648.402571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8648.713305] Lustre: lustre-OST0000: new disk, initializing [ 8648.716574] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8648.725582] Lustre: 259871:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8650.624841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8650.661395] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8650.799701] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8657.330515] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8670.908153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8671.086190] Lustre: lustre-OST0001: new disk, initializing [ 8671.089540] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8671.095269] Lustre: 260744:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8672.425821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8672.443446] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8672.615402] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8680.327683] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8696.193689] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:00:38 (1790748038) [ 8715.235804] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:00:58 (1790748058) [ 8732.642624] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8732.658846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8732.669902] Lustre: Skipped 5 previous similar messages [ 8732.682839] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8732.697527] Lustre: Skipped 3 previous similar messages [ 8736.720045] Lustre: server umount lustre-OST0000 complete [ 8743.905091] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8743.914618] LustreError: Skipped 1 previous similar message [ 8749.763645] Lustre: server umount lustre-MDT0000 complete [ 8754.478774] LustreError: 258281:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748099 with bad export cookie 12204917029467776682 [ 8754.491316] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8754.499392] LustreError: 258281:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8761.230570] Lustre: server umount lustre-MDT0001 complete [ 8774.151437] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 8850.864881] Lustre: server umount lustre-OST0001 complete [ 8858.617761] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 8874.605908] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 8913.982480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8914.259448] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8914.292458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8914.372970] Lustre: lustre-MDT0000: new disk, initializing [ 8914.439094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8914.446377] Lustre: Skipped 3 previous similar messages [ 8914.457317] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8920.002314] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8933.195487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8933.313477] Lustre: 265614:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8933.376559] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8933.382361] Lustre: Skipped 1 previous similar message [ 8933.522807] Lustre: lustre-MDT0001: new disk, initializing [ 8933.748214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8933.768203] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8939.808921] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8950.364429] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8956.333556] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8963.830117] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8964.091246] Lustre: 267583:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8965.387372] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8965.574554] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8972.265800] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 8983.727868] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8996.324392] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8996.339061] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8996.341767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8996.363859] Lustre: Skipped 6 previous similar messages [ 8996.378032] Lustre: Skipped 9 previous similar messages [ 8999.263387] Lustre: server umount lustre-OST0000 complete [ 9019.360481] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9019.726232] Lustre: server umount lustre-MDT0000 complete [ 9021.928578] LustreError: 267189:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9021.951189] LustreError: 267189:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 18 previous similar messages [ 9024.234494] LustreError: 265608:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748369 with bad export cookie 12204917029467778145 [ 9024.247709] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9024.254207] LustreError: 265608:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9024.674714] Lustre: server umount lustre-MDT0001 complete [ 9038.324975] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 9041.449357] Key type lgssc unregistered [ 9041.921302] LNet: 269343:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9041.933286] LNetError: 269343:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9041.970547] LNet: Removed LNI 192.168.204.136@tcp [ 9043.102181] Key type .llcrypt unregistered [ 9043.104084] Key type ._llcrypt unregistered [ 9073.762376] Key type ._llcrypt registered [ 9073.765127] Key type .llcrypt registered [ 9073.865488] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 9075.191182] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9075.251780] alg: No test for adler32 (adler32-zlib) [ 9076.322867] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 9076.580328] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 9078.328772] Key type lgssc registered [ 9079.798809] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9089.843922] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9089.860588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9091.310219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9095.612692] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9101.808774] LustreError: 270825:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9101.833340] LustreError: 270825:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9106.914577] LustreError: 270824:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9108.048221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9108.780662] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9115.316767] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9124.020545] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9131.992366] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9139.980879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9140.544182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9147.722449] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9155.874254] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9156.895476] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9159.137394] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9159.154076] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9159.187685] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9161.159687] Lustre: server umount lustre-OST0000 complete [ 9166.307538] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9166.311333] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9166.324228] Lustre: Skipped 1 previous similar message [ 9166.333263] Lustre: Skipped 1 previous similar message [ 9171.427312] LustreError: 271553:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9171.441268] LustreError: 271553:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9171.716467] Lustre: server umount lustre-MDT0000 complete [ 9175.867837] LustreError: 270806:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748520 with bad export cookie 17176734493241322087 [ 9175.873913] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9175.884333] LustreError: 270806:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9176.385903] Lustre: server umount lustre-MDT0001 complete [ 9188.157887] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 9190.727727] Key type lgssc unregistered [ 9191.122853] LNet: 274408:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9191.134861] LNetError: 274408:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9191.174936] LNet: Removed LNI 192.168.204.136@tcp [ 9192.071504] Key type .llcrypt unregistered [ 9192.074940] Key type ._llcrypt unregistered [ 9208.322807] Key type ._llcrypt registered [ 9208.325259] Key type .llcrypt registered [ 9208.437187] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 9209.733710] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9209.801691] alg: No test for adler32 (adler32-zlib) [ 9210.844701] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 9211.036353] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 9212.680312] Key type lgssc registered [ 9214.159373] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9225.761760] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9225.808966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9241.440848] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9246.560984] LustreError: 275618:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.136@tcp: failed processing log, type 4: rc = -110 [ 9273.248231] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9280.805214] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9291.181644] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9295.372281] Lustre: server umount lustre-OST0000 complete [ 9315.724989] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 9319.366409] Key type lgssc unregistered [ 9319.676128] LNet: 277249:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9319.683447] LNetError: 277249:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9319.695877] LNet: Removed LNI 192.168.204.136@tcp [ 9320.659303] Key type .llcrypt unregistered [ 9320.660884] Key type ._llcrypt unregistered [ 9371.260374] Key type ._llcrypt registered [ 9371.266340] Key type .llcrypt registered [ 9371.378801] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 9372.719820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9372.846066] alg: No test for adler32 (adler32-zlib) [ 9374.008229] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 9374.269348] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 9375.960251] Key type lgssc registered [ 9377.142735] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9390.700965] loop0: detected capacity change from 0 to 204800000 [ 9390.937861] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9390.950482] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9391.379810] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9392.320771] loop0: detected capacity change from 0 to 204800000 [ 9392.547079] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9392.564915] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9392.934695] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9394.187283] loop0: detected capacity change from 0 to 204800000 [ 9394.450787] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9394.477228] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9394.693947] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9395.934682] loop0: detected capacity change from 0 to 204800000 [ 9396.169781] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9396.177717] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9396.321971] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9397.511981] loop0: detected capacity change from 0 to 204800000 [ 9397.556957] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9402.722483] loop0: detected capacity change from 0 to 204800000 [ 9402.767138] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9406.805757] loop0: detected capacity change from 0 to 204800000 [ 9406.836106] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9411.028840] loop0: detected capacity change from 0 to 204800000 [ 9411.054788] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9415.079965] loop0: detected capacity change from 0 to 204800000 [ 9415.152118] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9415.225607] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9415.327869] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9415.341655] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9415.355732] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9415.381508] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9415.398813] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9415.415582] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9415.436581] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9415.449186] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9415.462844] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9415.481652] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9415.496408] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9415.510109] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9415.544484] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9415.562076] Lustre: 280686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9421.442421] Lustre: server umount MGS complete [ 9422.276112] loop0: detected capacity change from 0 to 204800000 [ 9422.382940] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9422.896523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9422.959392] Lustre: lustre-MDT0000: Aborting client recovery [ 9423.980489] loop1: detected capacity change from 0 to 204800000 [ 9424.058617] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9424.145993] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9424.171550] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9424.204641] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9424.215589] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9424.232494] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9424.243365] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9424.260853] Lustre: 281116:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9424.277960] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9424.472380] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9424.523104] Lustre: lustre-MDT0001: Aborting client recovery [ 9425.708693] loop2: detected capacity change from 0 to 204800000 [ 9425.882265] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9426.030815] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9426.043108] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9426.074085] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9426.088380] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9426.101639] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9426.117687] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9426.144551] Lustre: 281195:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9426.333783] Lustre: Found index 0 for lustre-OST0000, updating log [ 9426.379226] Lustre: 281214:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9426.469767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9426.493323] Lustre: lustre-OST0000: Aborting recovery [ 9427.452537] loop3: detected capacity change from 0 to 204800000 [ 9427.617248] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9427.708519] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9427.727781] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9427.741305] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9427.759732] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9427.772608] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9427.780996] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9427.792420] Lustre: 281278:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9427.809701] Lustre: Found index 1 for lustre-OST0001, updating log [ 9427.831713] Lustre: 281283:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9427.918539] Lustre: lustre-OST0001: Aborting recovery [ 9433.006466] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9433.009158] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9433.081383] LustreError: 281060:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9433.108193] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9433.161675] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9435.062100] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9435.065338] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9436.146175] LustreError: 281027:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2362us: rc = -2 [ 9436.649398] LustreError: 281028:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 596us: rc = -2 [ 9436.667606] LustreError: 281028:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 9449.954594] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9449.968531] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9449.994355] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9455.585661] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9455.593570] Lustre: Skipped 4 previous similar messages [ 9457.976378] Lustre: server umount lustre-MDT0000 complete [ 9458.994892] LustreError: 281012:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748803 with bad export cookie 12557694622869921061 [ 9459.032632] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9460.707285] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9460.708891] LustreError: 281025:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9460.709886] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9460.713542] Lustre: Skipped 4 previous similar messages [ 9460.728840] LustreError: 281025:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9463.712677] LustreError: 281564:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9463.714564] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9463.732648] LustreError: 281564:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9463.741030] Lustre: Skipped 2 previous similar messages [ 9465.469985] Lustre: server umount lustre-MDT0001 complete [ 9473.040653] Lustre: server umount lustre-OST0000 complete [ 9480.498333] Lustre: server umount lustre-OST0001 complete [ 9489.109177] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 02:13:52 (1790748832) [ 9510.434670] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 9550.075574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9550.341798] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9550.360569] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9550.413222] Lustre: lustre-MDT0000: new disk, initializing [ 9550.490865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9550.493247] Lustre: Skipped 1 previous similar message [ 9550.520475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9554.505928] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9562.338144] Lustre: server umount lustre-MDT0000 complete [ 9582.383780] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:15:25 (1790748925) [ 9611.006423] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 9620.135359] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 9653.191043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9653.528421] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9653.568960] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9653.572490] Lustre: Skipped 1 previous similar message [ 9653.638731] Lustre: lustre-MDT0000: new disk, initializing [ 9653.710389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9653.726758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9658.294433] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9668.664278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9668.808559] Lustre: 288387:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9668.875680] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9668.881454] Lustre: Skipped 1 previous similar message [ 9668.961479] Lustre: lustre-MDT0001: new disk, initializing [ 9669.049204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9669.080690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9669.090644] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9673.752524] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9682.475696] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9687.837282] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9694.641994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9694.858478] Lustre: lustre-OST0000: new disk, initializing [ 9694.861766] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9694.866462] Lustre: 290358:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9694.938341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9699.942231] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9702.453855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9702.462113] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9702.543193] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9707.756611] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9712.405559] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9725.113368] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9733.009964] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9774.562081] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9774.563331] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9774.575864] LustreError: Skipped 1 previous similar message [ 9774.576544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9774.587198] Lustre: Skipped 1 previous similar message [ 9774.598939] Lustre: Skipped 1 previous similar message [ 9777.546452] Lustre: server umount lustre-OST0000 complete [ 9781.734749] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9781.746196] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9784.805068] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9784.814196] Lustre: Skipped 2 previous similar messages [ 9787.878958] Lustre: server umount lustre-MDT0000 complete [ 9789.921270] LustreError: 288399:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9789.950693] LustreError: 288399:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9792.823385] LustreError: 288381:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790749137 with bad export cookie 12557694622869924029 [ 9792.830191] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9792.836797] LustreError: 288381:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9793.380274] Lustre: server umount lustre-MDT0001 complete [ 9803.850819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9809.278469] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9814.498211] Lustre: server umount MGS complete [ 9826.315754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9827.066885] LustreError: 294317:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9827.100735] LustreError: 294330:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9827.154432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9832.158428] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9832.432451] LustreError: 294318:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9840.786867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9846.342955] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9854.511807] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9860.045636] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9867.490797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9875.312667] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 9883.846638] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9913.826899] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9913.852141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9913.870083] Lustre: Skipped 1 previous similar message [ 9913.877178] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9918.669020] Lustre: server umount lustre-OST0000 complete [ 9923.046101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9923.057638] LustreError: Skipped 1 previous similar message [ 9923.062053] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9923.082264] Lustre: Skipped 1 previous similar message [ 9928.632300] Lustre: server umount lustre-MDT0000 complete [ 9931.655225] LustreError: 294299:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790749276 with bad export cookie 12557694622869925849 [ 9931.657803] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9931.670837] LustreError: 294299:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9932.015431] Lustre: server umount lustre-MDT0001 complete [ 9943.084280] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 9946.201593] Key type lgssc unregistered [ 9946.456072] LNet: 298263:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9946.468166] LNetError: 298263:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9946.493358] LNet: Removed LNI 192.168.204.136@tcp [ 9947.438876] Key type .llcrypt unregistered [ 9947.443338] Key type ._llcrypt unregistered [ 9981.256425] Key type ._llcrypt registered [ 9981.258759] Key type .llcrypt registered [ 9981.335912] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 9995.298364] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 9996.281841] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9996.491874] alg: No test for adler32 (adler32-zlib) [ 9997.548725] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [ 9997.976311] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 9999.648239] Key type lgssc registered [10000.765656] Lustre: Echo OBD driver; http://www.lustre.org/ [10034.530493] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10034.599239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10035.855231] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10035.900710] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10036.014562] Lustre: lustre-MDT0000: new disk, initializing [10036.103377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10036.133409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10040.409178] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10051.170675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10051.323812] Lustre: 302239:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10051.377906] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10051.384128] Lustre: Skipped 1 previous similar message [10051.498417] Lustre: lustre-MDT0001: new disk, initializing [10051.615650] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10051.671453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10051.685750] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10056.651465] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10065.389903] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10070.532639] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10076.641284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10076.866692] Lustre: lustre-OST0000: new disk, initializing [10076.869873] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10076.875707] Lustre: 304208:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10076.961971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10082.821556] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10083.904682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10083.930271] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10084.108847] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10093.247433] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10100.771858] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10109.751892] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10117.914536] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10122.638712] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10164.206354] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10164.217244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10164.221764] Lustre: Skipped 1 previous similar message [10164.234541] Lustre: Skipped 1 previous similar message [10169.318791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10169.326886] Lustre: Skipped 1 previous similar message [10170.189869] Lustre: server umount lustre-OST0000 complete [10174.441086] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10174.442746] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10174.452475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10174.467703] Lustre: Skipped 1 previous similar message [10179.553157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10179.566162] Lustre: Skipped 2 previous similar messages [10184.675325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10184.683941] Lustre: Skipped 1 previous similar message [10188.256309] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10188.402996] Lustre: server umount lustre-MDT0000 complete [10192.216282] LustreError: 302233:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790749537 with bad export cookie 3101795498941619576 [10192.218610] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10192.225067] LustreError: 302233:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [10192.740371] Lustre: server umount lustre-MDT0001 complete [10202.448053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10207.287509] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10212.102458] Lustre: server umount MGS complete [10224.015041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10224.493829] LustreError: 308079:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10224.531089] LustreError: 308079:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10224.533614] LustreError: 308092:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10224.614166] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10229.333214] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10229.742170] LustreError: 308080:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10234.851607] LustreError: 308079:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10237.865346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10238.386860] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10243.151392] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10252.303294] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10259.876757] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10267.863557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10268.328895] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10276.526754] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10284.193837] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10311.650802] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10311.656691] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10311.670126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10312.867578] Lustre: server umount lustre-OST0000 complete [10319.847425] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10319.862461] Lustre: Skipped 1 previous similar message [10323.434921] Lustre: server umount lustre-MDT0000 complete [10324.966945] LustreError: 310571:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10325.001249] LustreError: 310571:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10327.125588] LustreError: 310808:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790749672 with bad export cookie 3101795498941621508 [10327.138133] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10327.148984] LustreError: 310808:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [10338.185258] Lustre: server umount lustre-MDT0001 complete [10350.841856] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [10353.980380] Key type lgssc unregistered [10354.384583] LNet: 312026:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10354.392729] LNetError: 312026:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10354.414251] LNet: Removed LNI 192.168.204.136@tcp [10355.429826] Key type .llcrypt unregistered [10355.433859] Key type ._llcrypt unregistered [10376.998195] Key type ._llcrypt registered [10377.000367] Key type .llcrypt registered [10377.151625] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [10390.475929] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [10391.086190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10391.131882] alg: No test for adler32 (adler32-zlib) [10392.264172] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [10392.523068] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [10394.208427] Key type lgssc registered [10395.289180] Lustre: Echo OBD driver; http://www.lustre.org/ [10419.849945] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [10436.163866] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10436.236231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10437.639716] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10437.710534] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10437.826554] Lustre: lustre-MDT0000: new disk, initializing [10437.905122] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10437.942410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10442.630641] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [10454.635350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10454.892383] Lustre: lustre-OST0000: new disk, initializing [10454.897248] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10454.902541] Lustre: Skipped 1 previous similar message [10454.913699] Lustre: 316091:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10455.017918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10462.387628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10462.409589] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10462.468722] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10463.791083] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11312.612878] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11312.631777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11316.203700] Lustre: server umount lustre-MDT0000 complete [11320.534420] LustreError: 315280:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790750665 with bad export cookie 692900093092678003 [11320.547784] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11320.549181] LustreError: 315280:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [11320.748965] Lustre: server umount lustre-OST0000 complete [11349.071168] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [11360.330905] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [11399.304791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11399.595475] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11399.643324] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11399.731207] Lustre: lustre-MDT0000: new disk, initializing [11399.859104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11399.908579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11405.003707] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11418.827938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11419.015827] Lustre: 320444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11419.101329] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11419.104018] Lustre: Skipped 1 previous similar message [11419.219758] Lustre: lustre-MDT0001: new disk, initializing [11419.347880] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11419.402185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11419.414056] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11423.655989] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11433.194726] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11439.403279] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11446.858301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11447.242431] Lustre: lustre-OST0000: new disk, initializing [11447.252547] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11447.266768] Lustre: 322413:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11447.386549] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11453.576926] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11457.131112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11457.155524] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11457.303074] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11463.526500] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11468.606818] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11468.839824] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11473.873710] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11474.170832] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11477.986863] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11477.990283] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11478.008309] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11478.015091] Lustre: Skipped 1 previous similar message [11478.039710] Lustre: Skipped 1 previous similar message [11482.823938] Lustre: server umount lustre-OST0000 complete [11488.225703] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11488.229614] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11488.261032] Lustre: Skipped 1 previous similar message [11488.279593] Lustre: Skipped 1 previous similar message [11493.353155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11493.361052] Lustre: Skipped 1 previous similar message [11493.763502] Lustre: server umount lustre-MDT0000 complete [11498.632297] LustreError: 320437:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790750843 with bad export cookie 692900093095618822 [11498.635724] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11498.648233] LustreError: 320437:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [11498.944837] Lustre: server umount lustre-MDT0001 complete [11504.028916] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11506.308166] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 02:47:29 (1790750849) [11516.527885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11517.191222] LustreError: 324726:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11517.208265] LustreError: 324726:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11517.259653] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11522.538222] LustreError: 324727:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11522.747986] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11527.654843] LustreError: 324727:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11532.772058] LustreError: 324726:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11533.040778] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11533.487240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11539.064509] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11547.515534] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11554.107454] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11562.043948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11562.404324] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [11562.480280] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11570.051972] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11579.749381] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11587.677107] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11588.000506] Lustre: lustre-OST0001: new disk, initializing [11588.010643] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11588.043992] Lustre: 327563:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11588.283412] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11595.380412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11595.394604] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11595.445092] LustreError: 327893:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11595.676717] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11596.528543] LustreError: 327886:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11596.549289] LustreError: 327886:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11597.553641] LustreError: 327893:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11599.656440] LustreError: 327886:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11599.671625] LustreError: 327886:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11603.747405] LustreError: 327893:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11603.768525] LustreError: 327893:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11606.727109] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11607.058705] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11612.084974] LustreError: 327886:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11612.099851] LustreError: 327886:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11614.468568] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff990e42fd6000.ost_server_uuid 50 [11616.212074] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff990e42fd6000.ost_server_uuid in FULL state after 0 sec [11623.311031] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff990e42fd6000.ost_server_uuid 50 [11625.065344] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff990e42fd6000.ost_server_uuid in IDLE state after 0 sec [11633.147224] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11641.840324] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11641.876329] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11641.885264] Lustre: Skipped 1 previous similar message [11647.145640] Lustre: server umount lustre-OST0001 complete [11647.515611] LustreError: 326547:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11654.630221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11654.636399] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11654.644896] Lustre: Skipped 1 previous similar message [11655.656144] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11657.197488] LustreError: 326547:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11657.213275] LustreError: 326547:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11659.045269] Lustre: server umount lustre-OST0000 complete [11666.403093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11666.412269] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11666.433910] Lustre: Skipped 1 previous similar message [11666.444349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11666.453222] Lustre: Skipped 7 previous similar messages [11677.152187] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11677.397562] Lustre: server umount lustre-MDT0000 complete [11677.668296] LustreError: 325454:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11681.772688] LustreError: 324707:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751026 with bad export cookie 692900093095619704 [11681.788108] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11681.791922] LustreError: 324707:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [11682.300122] Lustre: server umount lustre-MDT0001 complete [11694.922431] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [11699.331824] Key type lgssc unregistered [11699.823226] LNet: 330253:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11699.833181] LNetError: 330253:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11699.859878] LNet: Removed LNI 192.168.204.136@tcp [11701.460391] Key type .llcrypt unregistered [11701.463255] Key type ._llcrypt unregistered [11729.315977] Key type ._llcrypt registered [11729.322535] Key type .llcrypt registered [11729.529902] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [11730.517072] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11730.766566] alg: No test for adler32 (adler32-zlib) [11731.859987] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [11732.107389] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [11733.779179] Key type lgssc registered [11734.976612] Lustre: Echo OBD driver; http://www.lustre.org/ [11741.876640] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11741.899370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11743.312037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11748.253561] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11753.443672] LustreError: 331688:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11753.458070] LustreError: 331688:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11755.420819] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11758.577991] LustreError: 331689:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11762.208158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11762.653207] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11767.628627] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11777.522179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11777.955209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11784.520441] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11787.176449] LustreError: 333369:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11788.226156] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11792.528208] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11793.383696] LustreError: 333371:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11793.400674] LustreError: 333371:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11798.504372] LustreError: 333855:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11798.534236] LustreError: 333855:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11800.179770] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11800.432325] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11805.708877] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11806.031856] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [11813.778217] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11820.801599] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11821.024136] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11928.544995] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11928.549357] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11928.557293] LustreError: Skipped 1 previous similar message [11928.570879] Lustre: Skipped 1 previous similar message [11928.570951] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11928.584756] Lustre: Skipped 1 previous similar message [11929.848826] Lustre: server umount lustre-OST0001 complete [11933.670874] LustreError: 333370:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11933.691595] LustreError: 333370:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11935.713523] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11935.720425] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11935.743554] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11936.737439] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11936.742803] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11936.759916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11938.794701] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11938.806434] Lustre: Skipped 1 previous similar message [11941.593243] Lustre: server umount lustre-OST0000 complete [11946.978946] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11946.983077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11946.990820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11959.264313] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11959.271582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11959.287325] Lustre: Skipped 5 previous similar messages [11959.545580] Lustre: server umount lustre-MDT0000 complete [11963.250833] LustreError: 331669:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751308 with bad export cookie 2889805761783143499 [11963.258588] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11963.265263] LustreError: 331669:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [11963.715987] Lustre: server umount lustre-MDT0001 complete [11973.865897] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [11976.594600] Key type lgssc unregistered [11976.935922] LNet: 336634:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11976.947299] LNetError: 336634:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11976.970432] LNet: Removed LNI 192.168.204.136@tcp [11977.889184] Key type .llcrypt unregistered [11977.893920] Key type ._llcrypt unregistered [12017.940792] Key type ._llcrypt registered [12017.944605] Key type .llcrypt registered [12018.129266] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [12019.724166] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12019.802880] alg: No test for adler32 (adler32-zlib) [12020.966470] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [12021.252466] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [12022.960748] Key type lgssc registered [12024.359761] Lustre: Echo OBD driver; http://www.lustre.org/ [12034.370834] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [12034.405948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12035.930245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12040.664653] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12045.292274] LustreError: 338946:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12045.312871] LustreError: 338946:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12049.609748] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12050.025585] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12055.397976] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12058.764756] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12065.368751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12065.779308] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12072.538642] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12074.995952] LustreError: 340433:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12080.123924] LustreError: 340436:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12080.156746] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12080.158551] LustreError: 340436:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12081.000338] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12081.247427] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12086.289853] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12089.577965] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12099.474700] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12106.883180] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12121.571952] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12121.577286] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12121.599492] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12122.086930] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12122.102132] Lustre: Skipped 1 previous similar message [12122.106498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12122.115269] Lustre: Skipped 3 previous similar messages [12124.024420] Lustre: server umount lustre-MDT0000 complete [12126.868300] LustreError: 338926:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751471 with bad export cookie 9591532966690126334 [12126.869341] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12126.880469] LustreError: 338926:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [12127.239622] Lustre: server umount lustre-MDT0001 complete [12141.112465] Lustre: server umount lustre-OST0000 complete [12143.392109] Lustre: 338200:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790751472/real 1790751472] req@ffffa0ddda259f80 x1877738904418816/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790751488 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12143.417172] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12143.944276] Lustre: server umount lustre-OST0001 complete [12158.533784] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [12167.957962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12168.254451] LustreError: 344208:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12168.321612] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12172.304355] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12173.796907] LustreError: 344209:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12180.000530] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12180.338507] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12184.060732] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12187.220596] Lustre: 345347:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12195.058434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12202.562779] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12205.750475] LustreError: 345705:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12205.780557] LustreError: 345705:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12211.467818] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12211.703396] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12211.714836] Lustre: Skipped 1 previous similar message [12213.793819] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12213.809525] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12217.721489] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12225.073653] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12228.389301] Lustre: 347221:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12231.653055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12231.659690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12231.671052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12234.217805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12234.223013] Lustre: Skipped 2 previous similar messages [12237.791250] Lustre: server umount lustre-MDT0000 complete [12239.338623] LustreError: 344939:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12239.367837] LustreError: 344939:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [12242.080254] LustreError: 345349:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751586 with bad export cookie 9591532966690128035 [12242.086661] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12242.093214] LustreError: 345349:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [12242.399974] Lustre: server umount lustre-MDT0001 complete [12256.870680] Lustre: server umount lustre-OST0000 complete [12260.721350] Lustre: 338201:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790751589/real 1790751589] req@ffffa0def7d97480 x1877738904466176/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790751605 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12260.759139] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12260.778905] Lustre: Skipped 3 previous similar messages [12262.094582] Lustre: server umount lustre-OST0001 complete [12266.905231] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12269.026330] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12271.328740] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:00:13 (1790751613) [12296.935823] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [12307.057495] loop0: detected capacity change from 0 to 1099511627776 [12307.734156] SGI XFS with ACLs, security attributes, quota, no debug enabled [12307.741926] *** VALIDATE xfs *** [12307.753602] XFS (loop0): Mounting V5 Filesystem [12307.814405] XFS (loop0): Ending clean mount [12309.107694] loop1: detected capacity change from 0 to 18691697672192 [12423.559626] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [12433.542727] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [12477.107126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12477.525700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12477.574060] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12477.715403] Lustre: lustre-MDT0000: new disk, initializing [12477.927734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12477.989986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12483.765177] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12496.843145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12497.050791] Lustre: 352369:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12497.125760] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12497.129924] Lustre: Skipped 1 previous similar message [12497.224662] Lustre: lustre-MDT0001: new disk, initializing [12497.441648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12497.461152] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12502.538988] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12513.554863] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12520.705616] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12529.474034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12529.778238] Lustre: lustre-OST0000: new disk, initializing [12529.786622] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12529.794355] Lustre: 354339:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12531.804030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12531.811451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12531.932956] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12537.312782] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12548.376630] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12553.115725] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12553.399907] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12558.877585] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12559.222435] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12562.915506] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12562.916844] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12562.924144] LustreError: Skipped 1 previous similar message [12562.925180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12562.934465] Lustre: Skipped 1 previous similar message [12562.947762] Lustre: Skipped 1 previous similar message [12567.909333] Lustre: server umount lustre-OST0000 complete [12573.169812] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12573.179797] Lustre: Skipped 1 previous similar message [12578.378714] Lustre: server umount lustre-MDT0000 complete [12582.233734] LustreError: 354338:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751927 with bad export cookie 9591532966690129582 [12582.239308] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12582.243384] LustreError: 354338:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [12582.741640] Lustre: server umount lustre-MDT0001 complete [12591.295047] XFS (loop0): Unmounting Filesystem [12596.812895] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:05:39 (1790751939) [12604.020243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12604.785678] LustreError: 356900:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12604.864644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12604.868738] Lustre: Skipped 2 previous similar messages [12609.901790] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12619.008172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12619.239259] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12619.252440] Lustre: Skipped 2 previous similar messages [12624.462916] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12632.346823] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12638.479203] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12646.885872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12655.336662] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12664.884651] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12675.044724] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12675.055179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12675.071341] Lustre: Skipped 2 previous similar messages [12675.085615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12680.171370] LustreError: 358720:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12680.199920] LustreError: 358720:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12680.283898] Lustre: server umount lustre-OST0000 complete [12690.321463] Lustre: server umount lustre-MDT0000 complete [12694.606310] LustreError: 356879:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790752039 with bad export cookie 9591532966690130464 [12694.614919] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12694.635096] LustreError: 356879:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [12695.104728] Lustre: server umount lustre-MDT0001 complete [12706.415984] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [12709.831826] Key type lgssc unregistered [12710.171735] LNet: 360528:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12710.179727] LNetError: 360528:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12711.211457] LNet: Removed LNI 192.168.204.136@tcp [12712.290495] Key type .llcrypt unregistered [12712.292385] Key type ._llcrypt unregistered [12745.173966] Key type ._llcrypt registered [12745.176459] Key type .llcrypt registered [12745.386282] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [12746.744320] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12746.875767] alg: No test for adler32 (adler32-zlib) [12748.236145] Lustre: Lustre: Build Version: 2.17.59_3_g21e240e [12748.624923] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [12750.312247] Key type lgssc registered [12751.653515] Lustre: Echo OBD driver; http://www.lustre.org/ [12760.983905] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [12761.035634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12762.748290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12767.707813] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12772.324638] LustreError: 362112:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12772.335029] LustreError: 362112:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12776.844237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12777.312495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12782.402621] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12789.922624] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12796.125440] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12803.324574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12803.662128] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12810.173402] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12817.806469] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12828.641784] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12828.652637] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12828.667744] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12830.781736] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.36@tcp (stopping) [12833.253928] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12833.255696] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12834.839633] Lustre: server umount lustre-MDT0001 complete [12835.862026] LustreError: 362109:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12838.373564] LustreError: 362112:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12838.389418] LustreError: 362112:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12840.986641] LustreError: 362107:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12845.757739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12845.886341] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12845.894374] Lustre: Found index 1 for lustre-MDT0001, updating log [12845.951477] Lustre: 362100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12846.131343] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.36@tcp (not set up) [12846.144849] Lustre: Skipped 1 previous similar message [12846.421822] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12851.285252] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12851.694609] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12853.266592] LustreError: 365646:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12853.280648] LustreError: 365646:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12853.292049] LustreError: 362104:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12853.391566] LustreError: 365657:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.136@tcp: cfg command failed: rc = -17 [12853.409832] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12853.409832] [12866.628638] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.36@tcp (stopping) [12867.041084] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12867.049867] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12867.067324] Lustre: Skipped 1 previous similar message [12868.718350] Lustre: server umount lustre-MDT0001 complete [12871.702091] LustreError: 362108:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12871.728855] LustreError: 362108:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12881.093524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12881.246920] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12881.254886] Lustre: Found index 1 for lustre-MDT0001, updating log [12881.307807] Lustre: 362100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12881.721701] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12886.919812] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [12887.021436] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12888.070696] LustreError: 366695:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12888.097111] LustreError: 366695:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12888.109183] LustreError: 362104:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12888.120731] LustreError: 362104:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12888.248760] LustreError: 366705:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12888.275069] LustreError: 366705:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12888.284881] LustreError: 366705:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.136@tcp: cfg command failed: rc = -17 [12888.303122] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.136@tcp [12888.303122] [13194.211751] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13194.231935] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13194.240059] Lustre: Skipped 3 previous similar messages [13197.650661] Lustre: server umount lustre-MDT0001 complete [13199.329962] LustreError: 362113:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13199.369925] LustreError: 362113:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [13209.358386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13209.426212] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13209.429777] Lustre: Found index 1 for lustre-MDT0001, updating log [13209.456439] Lustre: 362100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13209.683288] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13213.979597] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [13214.714899] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13216.758633] LustreError: 374949:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13216.774873] LustreError: 374949:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13216.783855] LustreError: 362104:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13216.791646] LustreError: 362104:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13216.915435] LustreError: 374966:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13216.929953] LustreError: 374966:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.136@tcp: cfg command failed: rc = -114 [13216.941377] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13216.941377] [13537.250483] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13537.258679] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13537.260327] Lustre: Skipped 1 previous similar message [13537.262994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13537.276019] Lustre: Skipped 5 previous similar messages [13542.656443] Lustre: server umount lustre-MDT0000 complete [13546.471089] LustreError: 365301:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790752891 with bad export cookie 4543592361129227086 [13546.472958] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13546.485379] LustreError: 365301:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [13547.489511] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13547.492793] LustreError: 363910:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13547.517501] Lustre: Skipped 2 previous similar messages [13547.536394] LustreError: 363910:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [13553.045356] Lustre: server umount lustre-MDT0001 complete [13557.430451] Lustre: server umount lustre-OST0000 complete [13567.964377] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [13577.874630] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [13615.817856] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:22:38 (1790752958) [13622.793862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13623.163994] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13623.213215] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13623.313480] Lustre: lustre-MDT0000: new disk, initializing [13623.370752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13623.382556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13628.090346] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [13638.829538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13638.935597] Lustre: 385799:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13638.985852] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13638.990067] Lustre: Skipped 1 previous similar message [13639.056426] Lustre: lustre-MDT0001: new disk, initializing [13639.168167] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13639.234593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13639.258577] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13643.910356] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [13656.421854] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13663.155407] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13671.881975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13672.154457] Lustre: lustre-OST0000: new disk, initializing [13672.161956] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13672.173464] Lustre: 387769:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13672.279175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13679.110457] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [13681.734502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13681.748244] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13681.856303] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13688.677997] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13697.508174] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13697.516373] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13697.534290] Lustre: Skipped 1 previous similar message [13697.539413] Lustre: Skipped 5 previous similar messages [13701.857863] Lustre: server umount lustre-MDT0000 complete [13702.634816] LustreError: 387169:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13702.655612] LustreError: 387169:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [13706.224796] LustreError: 385793:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790753051 with bad export cookie 4543592361129229039 [13706.243840] LustreError: 386607:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13706.246686] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13706.250554] LustreError: 385793:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [13706.788505] Lustre: server umount lustre-MDT0001 complete [13720.433371] Lustre: server umount lustre-OST0000 complete [13751.790372] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [13762.950353] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [13796.924340] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13543 sec ========= 03:25:39 (1790753139) [13799.166351] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:25:41 (1790753141) === [13803.115914] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:25:45 (1790753145) === [13845.474640] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [13848.538555] Key type lgssc unregistered [13848.902491] LNet: 394136:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13848.912104] LNetError: 394136:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13848.927119] LNet: Removed LNI 192.168.204.136@tcp [13849.905901] Key type .llcrypt unregistered [13849.914301] Key type ._llcrypt unregistered