[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 635315535 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002400] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007450] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.008029] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.009016] pid_max: default: 32768 minimum: 301 [ 0.010159] LSM: Security Framework initializing [ 0.011056] Yama: becoming mindful. [ 0.012045] SELinux: Initializing. [ 0.013110] *** VALIDATE selinux *** [ 0.020803] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025705] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026177] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027126] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029009] *** VALIDATE tmpfs *** [ 0.030490] *** VALIDATE proc *** [ 0.031262] *** VALIDATE cgroup *** [ 0.032012] *** VALIDATE cgroup2 *** [ 0.034221] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035174] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038029] Spectre V2 : User space: Vulnerable [ 0.039012] Speculative Store Bypass: Vulnerable [ 0.042388] debug: unmapping init [mem 0xffffffffb9e59000-0xffffffffb9e60fff] [ 0.044202] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045811] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046027] ... version: 2 [ 0.047018] ... bit width: 48 [ 0.048014] ... generic registers: 4 [ 0.049011] ... value mask: 0000ffffffffffff [ 0.050010] ... max period: 00007fffffffffff [ 0.051010] ... fixed-purpose events: 3 [ 0.052013] ... event mask: 000000070000000f [ 0.054272] rcu: Hierarchical SRCU implementation. [ 0.056661] smp: Bringing up secondary CPUs ... [ 0.057649] x86: Booting SMP configuration: [ 0.058034] .... node #0, CPUs: #1 #2 #3 [ 0.061476] smp: Brought up 1 node, 4 CPUs [ 0.063025] smpboot: Max logical packages: 1 [ 0.064021] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.255030] node 0 deferred pages initialised in 188ms [ 0.257111] devtmpfs: initialized [ 0.258167] x86/mm: Memory block size: 128MB [ 0.260683] gcov: version magic: 0x41383552 [ 0.263341] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.267080] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.270351] pinctrl core: initialized pinctrl subsystem [ 0.272144] [ 0.272534] ************************************************************* [ 0.275010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.277011] ** ** [ 0.280012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.282010] ** ** [ 0.285012] ** This means that this kernel is built to expose internal ** [ 0.288020] ** IOMMU data structures, which may compromise security on ** [ 0.290010] ** your system. ** [ 0.293013] ** ** [ 0.296013] ** If you see this message and you are not debugging the ** [ 0.299010] ** kernel, report this immediately to your vendor! ** [ 0.302011] ** ** [ 0.304010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.306011] ************************************************************* [ 0.309646] NET: Registered protocol family 16 [ 0.311416] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.314052] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.316053] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.321157] cpuidle: using governor menu [ 0.322764] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.325562] PCI: Using configuration type 1 for base access [ 0.327143] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.337084] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.339013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.342091] cryptd: max_cpu_qlen set to 1000 [ 0.346223] ACPI: Added _OSI(Module Device) [ 0.347015] ACPI: Added _OSI(Processor Device) [ 0.350015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.352010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.356861] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.363572] ACPI: Interpreter enabled [ 0.364066] ACPI: PM: (supports S0 S3 S4 S5) [ 0.366012] ACPI: Using IOAPIC for interrupt routing [ 0.367113] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.370414] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.382449] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.385042] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.387017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.389092] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.393392] acpiphp: Slot [2] registered [ 0.395117] acpiphp: Slot [5] registered [ 0.396131] acpiphp: Slot [6] registered [ 0.398141] acpiphp: Slot [7] registered [ 0.399133] acpiphp: Slot [8] registered [ 0.400099] acpiphp: Slot [9] registered [ 0.402120] acpiphp: Slot [10] registered [ 0.403115] acpiphp: Slot [3] registered [ 0.405070] acpiphp: Slot [4] registered [ 0.406108] acpiphp: Slot [11] registered [ 0.407078] acpiphp: Slot [12] registered [ 0.408079] acpiphp: Slot [13] registered [ 0.409082] acpiphp: Slot [14] registered [ 0.411072] acpiphp: Slot [15] registered [ 0.412070] acpiphp: Slot [16] registered [ 0.413070] acpiphp: Slot [17] registered [ 0.414120] acpiphp: Slot [18] registered [ 0.416100] acpiphp: Slot [19] registered [ 0.417073] acpiphp: Slot [20] registered [ 0.418082] acpiphp: Slot [21] registered [ 0.420091] acpiphp: Slot [22] registered [ 0.421080] acpiphp: Slot [23] registered [ 0.422112] acpiphp: Slot [24] registered [ 0.425116] acpiphp: Slot [25] registered [ 0.426116] acpiphp: Slot [26] registered [ 0.427079] acpiphp: Slot [27] registered [ 0.429055] acpiphp: Slot [28] registered [ 0.430095] acpiphp: Slot [29] registered [ 0.431077] acpiphp: Slot [30] registered [ 0.433070] acpiphp: Slot [31] registered [ 0.434073] PCI host bridge to bus 0000:00 [ 0.436016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.438017] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.440017] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.442016] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.444021] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.449024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.451179] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.453129] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.456366] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.464013] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.469058] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.472017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.474013] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.477016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.479549] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.482708] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.484031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.487620] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.492014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.505016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.509012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.513525] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.547021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.564017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.579014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.588683] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.621016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.653015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.680017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.710593] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.724015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.748016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.812017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.841658] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.855014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.870016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.913014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.927875] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.938014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.949016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.989018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.006159] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.028014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.064019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.148011] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.178000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.182316] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.185332] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.187348] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.190256] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.196047] iommu: Default domain type: Passthrough [ 1.197400] SCSI subsystem initialized [ 1.199134] ACPI: bus type USB registered [ 1.201083] usbcore: registered new interface driver usbfs [ 1.203065] usbcore: registered new interface driver hub [ 1.205160] usbcore: registered new device driver usb [ 1.207213] pps_core: LinuxPPS API ver. 1 registered [ 1.209010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.213062] PTP clock support registered [ 1.215081] EDAC MC: Ver: 3.0.0 [ 1.217109] PCI: Using ACPI for IRQ routing [ 1.218698] NetLabel: Initializing [ 1.220010] NetLabel: domain hash size = 128 [ 1.222012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.225080] NetLabel: unlabeled traffic allowed by default [ 1.227119] vgaarb: loaded [ 1.229225] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.230012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.237011] clocksource: Switched to clocksource kvm-clock [ 1.339144] VFS: Disk quotas dquot_6.6.0 [ 1.340897] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.343557] *** VALIDATE ramfs *** [ 1.344834] *** VALIDATE hugetlbfs *** [ 1.346624] pnp: PnP ACPI init [ 1.348599] pnp: PnP ACPI: found 6 devices [ 1.367171] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.370345] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.372441] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.374351] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.376424] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.378839] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.381432] NET: Registered protocol family 2 [ 1.383671] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.387946] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.391275] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.396254] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.399502] TCP: Hash tables configured (established 65536 bind 65536) [ 1.402207] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.404972] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.407419] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.410530] NET: Registered protocol family 1 [ 1.412550] RPC: Registered named UNIX socket transport module. [ 1.414373] RPC: Registered udp transport module. [ 1.415563] RPC: Registered tcp transport module. [ 1.416730] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.418620] NET: Registered protocol family 44 [ 1.419701] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.421126] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.422549] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.424177] PCI: CLS 0 bytes, default 64 [ 1.426306] Unpacking initramfs... [ 2.766553] debug: unmapping init [mem 0xffff9600fcc54000-0xffff9600fffbffff] [ 2.770282] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.772360] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.775560] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.272225] Initialise system trusted keyrings [ 3.275162] Key type blacklist registered [ 3.277487] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.286965] zbud: loaded [ 3.290057] *** VALIDATE nfs *** [ 3.291436] *** VALIDATE nfs4 *** [ 3.292959] pstore: using deflate compression [ 3.301678] Platform Keyring initialized [ 3.431488] NET: Registered protocol family 38 [ 3.432776] Key type asymmetric registered [ 3.433771] Asymmetric key parser 'x509' registered [ 3.435311] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.438409] io scheduler mq-deadline registered [ 3.440045] io scheduler kyber registered [ 3.441759] io scheduler bfq registered [ 3.444462] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.447299] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.450135] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.452884] ACPI: Power Button [PWRF] [ 3.458260] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.464836] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.508282] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.539707] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.616197] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.643463] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.672571] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.676766] Non-volatile memory driver v1.3 [ 3.678199] Linux agpgart interface v0.103 [ 3.707200] virtio_blk virtio1: [vda] 145872 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.709401] vda: detected capacity change from 0 to 74686464 [ 3.725911] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.728230] vdb: detected capacity change from 0 to 1073741824 [ 3.747334] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.751687] vdc: detected capacity change from 0 to 2621440000 [ 3.770038] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.774061] vdd: detected capacity change from 0 to 2621440000 [ 3.793759] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.797982] vde: detected capacity change from 0 to 4294967296 [ 3.817986] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.821262] vdf: detected capacity change from 0 to 4294967296 [ 3.831928] libphy: Fixed MDIO Bus: probed [ 3.838125] usbcore: registered new interface driver usbserial_generic [ 3.840739] usbserial: USB Serial support registered for generic [ 3.843631] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.847707] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.849788] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.852547] mousedev: PS/2 mouse device common for all mice [ 3.856121] rtc_cmos 00:05: RTC can wake from S4 [ 3.859204] rtc_cmos 00:05: registered as rtc0 [ 3.861320] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.862222] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.866601] intel_pstate: CPU model not supported [ 3.869477] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.871806] hid: raw HID events driver (C) Jiri Kosina [ 3.874859] usbcore: registered new interface driver usbhid [ 3.876422] usbhid: USB HID core driver [ 3.878093] drop_monitor: Initializing network drop monitor service [ 3.879883] Initializing XFRM netlink socket [ 3.881735] NET: Registered protocol family 10 [ 3.884158] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.884737] Segment Routing with IPv6 [ 3.890014] NET: Registered protocol family 17 [ 3.891803] mpls_gso: MPLS GSO support [ 3.898202] RAS: Correctable Errors collector initialized. [ 3.900857] AVX version of gcm_enc/dec engaged. [ 3.903035] AES CTR mode by8 optimization enabled [ 3.988090] sched_clock: Marking stable (3988065695, 0)->(4888440962, -900375267) [ 3.991303] registered taskstats version 1 [ 3.993339] Loading compiled-in X.509 certificates [ 3.995468] zswap: loaded using pool lzo/zbud [ 4.017342] Key type big_key registered [ 4.030046] Key type encrypted registered [ 4.031852] ima: No TPM chip found, activating TPM-bypass! [ 4.034177] ima: Allocated hash algorithm: sha1 [ 4.035895] ima: No architecture policies found [ 4.037655] evm: Initialising EVM extended attributes: [ 4.039603] evm: security.selinux [ 4.040843] evm: security.ima [ 4.041945] evm: security.capability [ 4.043332] evm: HMAC attrs: 0x1 [ 4.046529] rtc_cmos 00:05: setting system clock to 2026-08-03 19:43:47 UTC (1785786227) [ 4.053476] debug: unmapping init [mem 0xffffffffbae03000-0xffffffffbaffffff] [ 4.056557] debug: unmapping init [mem 0xffffffffb9b82000-0xffffffffb9e58fff] [ 4.066126] Write protecting the kernel read-only data: 28672k [ 4.069546] debug: unmapping init [mem 0xffffffffb8203000-0xffffffffb83fffff] [ 4.072344] debug: unmapping init [mem 0xffffffffb8b14000-0xffffffffb8bfffff] [ 4.107247] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.116163] systemd[1]: Detected virtualization kvm. [ 4.118153] systemd[1]: Detected architecture x86-64. [ 4.120445] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.146367] systemd[1]: No hostname configured. [ 4.148376] systemd[1]: Set hostname to . [ 4.150627] random: systemd: uninitialized urandom read (16 bytes read) [ 4.153382] systemd[1]: Initializing machine ID from random generator. [ 4.206660] random: ln: uninitialized urandom read (6 bytes read) [ 4.286749] random: systemd: uninitialized urandom read (16 bytes read) [ 4.289584] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 4.294537] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.298722] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Reached target Timers. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.862561] device-mapper: uevent: version 1.0.3 [ 4.865097] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.538659] virtio_net virtio0 ens2: renamed from eth0 [ 5.552426] random: fast init done [ 5.582560] scsi host0: ata_piix [ 5.586427] scsi host1: ata_piix [ 5.588909] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.591777] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.973483] dracut-initqueue[585]: RTNETLINK answers: File exists [ 10.548870] random: crng init done [ 10.550366] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.912299] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.987205] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.357161] SELinux: Disabled at runtime. [ 12.427266] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.438153] systemd[1]: Detected virtualization kvm. [ 12.439927] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.945900] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.948507] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.952931] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.955643] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.957881] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.964711] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.972064] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Apply Kernel Variables... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice User and Session Slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd Root File System. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 13.137904] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.435607] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.739690] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.758938] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.866691] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.881170] EDAC sbridge: Ver: 1.1.2 [ 15.889315] Key type dns_resolver registered [ 16.218553] NFS: Registering the id_resolver key type [ 16.219989] Key type id_resolver registered [ 16.221068] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg441-server login: [ 72.230460] libcfs: loading out-of-tree module taints kernel. [ 72.324707] Key type ._llcrypt registered [ 72.328614] Key type .llcrypt registered [ 72.530922] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 101.971042] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 104.049505] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 104.085637] alg: No test for adler32 (adler32-zlib) [ 105.683588] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 107.237549] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 109.184264] Key type lgssc registered [ 112.498583] Lustre: Echo OBD driver; http://www.lustre.org/ [ 138.210269] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 196.458260] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 214.058587] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 214.133306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 215.638869] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 215.702436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 215.813542] Lustre: lustre-MDT0000: new disk, initializing [ 215.917844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 215.957771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 223.551072] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 240.807101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 241.074905] Lustre: 6513:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 241.181913] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 241.189458] Lustre: Skipped 1 previous similar message [ 241.275621] Lustre: lustre-MDT0001: new disk, initializing [ 241.342929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 241.367613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 241.387143] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 248.168136] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 255.038959] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 269.576892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 269.934371] Lustre: lustre-OST0000: new disk, initializing [ 269.937811] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 269.941951] Lustre: 8452:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 270.048083] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 278.741158] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 279.138066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 279.153928] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 279.218373] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 299.181699] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 299.415553] Lustre: lustre-OST0001: new disk, initializing [ 299.420840] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 299.428255] Lustre: 9529:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 299.506702] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 308.869579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 308.901819] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 309.105066] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 313.564510] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 331.309512] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 338.881546] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 347.717303] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing check_logdir /tmp/testlogs/ [ 354.632640] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing yml_node [ 360.497878] Lustre: DEBUG MARKER: Client: 2.17.56.1 [ 364.693405] Lustre: DEBUG MARKER: MDS: 2.17.56.1 [ 368.132916] Lustre: DEBUG MARKER: OSS: 2.17.56.1 [ 371.021435] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 3 15:49:51 EDT 2026 [ 391.279205] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 393.358588] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 411.615676] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 411.617574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 411.637615] Lustre: Skipped 1 previous similar message [ 411.674804] Lustre: Skipped 3 previous similar messages [ 416.736815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 416.757258] Lustre: Skipped 3 previous similar messages [ 418.581540] Lustre: server umount lustre-MDT0000 complete [ 423.773498] LustreError: 6504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785786647 with bad export cookie 11736253020582306495 [ 423.773898] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 423.786069] LustreError: 6504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 426.976281] LustreError: 8824:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 426.980767] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 426.993465] LustreError: 8824:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 427.014017] Lustre: Skipped 2 previous similar messages [ 427.020601] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 427.026079] Lustre: Skipped 1 previous similar message [ 429.025313] LustreError: 6520:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 429.026926] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 429.054259] Lustre: Skipped 1 previous similar message [ 434.143838] LustreError: 6520:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 434.144164] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 434.173447] LustreError: 6520:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 434.206661] Lustre: Skipped 1 previous similar message [ 438.239250] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 439.137934] Lustre: server umount lustre-MDT0001 complete [ 444.390856] Lustre: server umount lustre-OST0000 complete [ 449.149915] Lustre: server umount lustre-OST0001 complete [ 458.799256] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 472.349657] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 517.832861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 518.200594] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 518.249409] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 518.332318] Lustre: lustre-MDT0000: new disk, initializing [ 518.425827] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 518.447849] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 525.196470] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 539.009221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 539.381607] Lustre: 15087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 539.521924] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 539.527815] Lustre: Skipped 1 previous similar message [ 539.672579] Lustre: lustre-MDT0001: new disk, initializing [ 539.873501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 539.976046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 539.998904] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 544.940833] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 558.570196] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 566.238725] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 574.920605] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 575.357663] Lustre: lustre-OST0000: new disk, initializing [ 575.365646] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 575.374068] Lustre: 17060:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 575.471459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 577.266076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 577.308803] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 577.480189] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 583.639993] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 596.859616] hrtimer: interrupt took 2726939 ns [ 597.403770] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 604.120708] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 604.529140] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 611.153773] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 611.398508] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 618.464802] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 618.467326] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 618.483142] LustreError: Skipped 1 previous similar message [ 618.485631] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 618.541016] Lustre: Skipped 2 previous similar messages [ 620.564320] Lustre: server umount lustre-OST0000 complete [ 627.170331] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 627.193090] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 633.848168] LustreError: 16459:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 633.879279] LustreError: 16459:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 634.121237] Lustre: server umount lustre-MDT0000 complete [ 639.163700] LustreError: 15100:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 639.200850] LustreError: 15100:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 639.643077] LustreError: 15082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785786863 with bad export cookie 11736253020582308028 [ 639.678308] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 640.357882] Lustre: server umount lustre-MDT0001 complete [ 651.421511] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:54:30 (1785786870) [ 662.917468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 663.603358] LustreError: 19332:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 663.727320] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 669.851085] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 681.785036] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 688.581873] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 700.725457] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 710.607942] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 723.335589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 733.100849] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 747.854744] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 753.997802] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 764.384618] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 764.393680] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 764.417925] Lustre: Skipped 1 previous similar message [ 764.437874] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 764.447057] Lustre: Skipped 4 previous similar messages [ 769.555631] Lustre: server umount lustre-MDT0000 complete [ 770.018696] LustreError: 20769:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 770.045714] LustreError: 20769:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 774.962104] LustreError: 21871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785786998 with bad export cookie 11736253020582308910 [ 774.977710] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 774.982179] LustreError: 21871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 775.142154] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 775.154884] Lustre: Skipped 2 previous similar messages [ 775.669621] Lustre: server umount lustre-MDT0001 complete [ 781.757955] Lustre: server umount lustre-OST0000 complete [ 821.381270] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 841.321052] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 843.287482] LustreError: 23745:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 843.343594] LustreError: 23745:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 843.581802] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 843.584009] Lustre: Skipped 2 previous similar messages [ 852.468114] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 869.085636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 878.129794] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 882.027859] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 891.538839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 901.937486] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 913.783444] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 914.189583] Lustre: lustre-OST0001: new disk, initializing [ 914.199869] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 914.212347] Lustre: 26066:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 915.514653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 915.544416] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 915.694954] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 922.596915] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 933.766944] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 938.583794] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 956.919950] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 956.935359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 956.940470] Lustre: Skipped 3 previous similar messages [ 956.965076] Lustre: Skipped 6 previous similar messages [ 961.945809] Lustre: server umount lustre-MDT0000 complete [ 962.046209] LustreError: 23740:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 962.079029] LustreError: 23740:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 967.226777] LustreError: 23727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785787190 with bad export cookie 11736253020582309757 [ 967.235115] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 967.241550] LustreError: 23727:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 967.846586] Lustre: server umount lustre-MDT0001 complete [ 983.302660] Lustre: server umount lustre-OST0000 complete [ 988.127257] Lustre: 3641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785787195/real 1785787195] req@ffff96017d3ec000 x1872532687046656/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785787211 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 989.426530] Lustre: server umount lustre-OST0001 complete [ 1015.608556] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 1031.375736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1032.401531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1032.406984] Lustre: Skipped 3 previous similar messages [ 1039.784575] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1052.279679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1059.346848] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1063.701280] Lustre: 30250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1072.938707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1082.639418] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1094.133344] LustreError: 30605:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1094.158561] LustreError: 30605:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 16 previous similar messages [ 1094.766228] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1103.265254] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1112.430113] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1116.851617] Lustre: 32127:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1124.320378] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1124.342181] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1124.374617] Lustre: Skipped 1 previous similar message [ 1124.389558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1132.518805] Lustre: server umount lustre-MDT0000 complete [ 1137.013466] LustreError: 29069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785787360 with bad export cookie 11736253020582311052 [ 1137.020307] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1137.033992] LustreError: 29069:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1137.646379] Lustre: server umount lustre-MDT0001 complete [ 1151.884904] Lustre: server umount lustre-OST0000 complete [ 1166.729294] Lustre: server umount lustre-OST0001 complete [ 1178.792121] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:03:18 (1785787398) [ 1219.807531] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 1235.544289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1241.339702] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1251.251938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1257.555561] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1261.311918] Lustre: 36102:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1270.675664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1279.951083] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1292.428138] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1292.637151] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1292.646087] Lustre: Skipped 6 previous similar messages [ 1303.192488] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1313.017316] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1317.511425] Lustre: 37983:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1323.488241] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1323.497177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1323.505836] Lustre: Skipped 6 previous similar messages [ 1323.524069] Lustre: Skipped 6 previous similar messages [ 1329.830357] Lustre: server umount lustre-MDT0000 complete [ 1334.968331] LustreError: 34936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785787558 with bad export cookie 11736253020582312158 [ 1334.971050] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1334.985091] LustreError: 34936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1335.666697] Lustre: server umount lustre-MDT0001 complete [ 1351.829907] Lustre: server umount lustre-OST0000 complete [ 1354.719883] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785787562/real 1785787562] req@ffff96017f643100 x1872532687162624/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785787578 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1356.255831] Lustre: 3641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785787563/real 1785787563] req@ffff96017d3ed880 x1872532687162880/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785787579 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1357.707600] Lustre: server umount lustre-OST0001 complete [ 1370.090727] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:06:30 (1785787590) [ 1393.910495] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 1412.725785] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1413.459795] LustreError: 40146:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1413.487450] LustreError: 40146:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 25 previous similar messages [ 1419.964204] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1431.844483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1438.417795] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1442.189053] Lustre: 41293:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1450.442560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1459.424227] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1470.218641] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1478.784668] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1490.026538] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1496.501401] Lustre: 43173:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1498.345210] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1498.356932] Lustre: Skipped 1 previous similar message [ 1506.813805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1506.854454] Lustre: Skipped 4 previous similar messages [ 1511.856984] Lustre: server umount lustre-MDT0000 complete [ 1521.102607] LustreError: 40126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785787744 with bad export cookie 11736253020582313264 [ 1521.108750] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1521.170025] LustreError: 40126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1528.408867] Lustre: server umount lustre-MDT0001 complete [ 1544.308347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1554.646996] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1555.780528] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1568.711821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1574.927616] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1574.944654] Lustre: Skipped 1 previous similar message [ 1577.647555] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1589.846969] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1592.420508] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1599.993613] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1602.258984] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1610.727376] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1610.744532] Lustre: Skipped 14 previous similar messages [ 1612.841823] Lustre: server umount lustre-MDT0000 complete [ 1615.341218] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1619.723386] LustreError: 40128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785787843 with bad export cookie 11736253020582314419 [ 1619.725222] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1619.744503] LustreError: 40128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1620.738678] Lustre: server umount lustre-MDT0001 complete [ 1637.344085] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785787844/real 1785787844] req@ffff960049f54380 x1872532687268992/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785787860 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1637.450118] Lustre: server umount lustre-OST0000 complete [ 1642.463224] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785787849/real 1785787849] req@ffff960177c8c000 x1872532687269504/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785787865 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1642.550198] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1644.380242] Lustre: server umount lustre-OST0001 complete [ 1661.666309] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:11:21 (1785787881) [ 1682.734986] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 1700.744953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1711.232185] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1723.476459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1731.728609] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1737.052454] Lustre: 48949:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1748.952288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1758.673290] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1771.249548] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1784.178777] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1793.631120] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1799.313418] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1819.107484] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1819.134372] Lustre: Skipped 8 previous similar messages [ 1824.349634] Lustre: server umount lustre-MDT0000 complete [ 1828.672229] LustreError: 47775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785788052 with bad export cookie 11736253020582315245 [ 1828.679368] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1828.690432] LustreError: 47775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1829.679479] Lustre: server umount lustre-MDT0001 complete [ 1836.144214] Lustre: server umount lustre-OST0000 complete [ 1840.843921] Lustre: server umount lustre-OST0001 complete [ 1853.382185] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:14:33 (1785788073) [ 1856.889121] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1859.095962] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:14:39 (1785788079) [ 1872.722339] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 1886.029691] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 1919.532610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1920.037614] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1920.238892] Lustre: lustre-MDT0000: new disk, initializing [ 1920.431359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1920.433996] Lustre: Skipped 10 previous similar messages [ 1920.454283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1926.306335] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1939.988808] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1948.726496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1949.214564] Lustre: lustre-OST0000: new disk, initializing [ 1949.225786] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1949.235037] Lustre: Skipped 1 previous similar message [ 1949.252954] Lustre: 55050:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1951.252723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1951.273819] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1951.359798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1961.101515] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 1973.017927] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1975.921456] Lustre: DEBUG MARKER: create test files [ 2017.777671] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2023.998661] Lustre: server umount lustre-OST0000 complete [ 2030.850383] Lustre: server umount lustre-MDT0000 complete [ 2044.952277] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 2048.930422] Key type lgssc unregistered [ 2049.279849] LNet: 56683:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2049.296660] LNetError: 56683:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2049.322137] LNet: Removed LNI 192.168.204.141@tcp [ 2050.773129] Key type .llcrypt unregistered [ 2050.775470] Key type ._llcrypt unregistered [ 2090.027386] Key type ._llcrypt registered [ 2090.033473] Key type .llcrypt registered [ 2090.191981] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 2092.093368] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2092.122072] alg: No test for adler32 (adler32-zlib) [ 2093.233558] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 2093.708637] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 2095.559173] Key type lgssc registered [ 2096.811880] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2106.894315] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2106.948590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2108.920264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2116.503060] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2125.860562] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2134.118236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2134.611398] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2139.638174] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 2139.737554] Lustre: 59420:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 2139.757867] Lustre: 59420:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 2139.770065] Lustre: 59420:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2139.782056] Lustre: 59420:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2139.791549] Lustre: 59420:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2139.802754] Lustre: 59420:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2141.665302] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2151.500366] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2156.941920] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2162.857134] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2175.460949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2175.466967] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2175.513814] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2180.890626] Lustre: server umount lustre-OST0000 complete [ 2190.174667] Lustre: server umount lustre-MDT0000 complete [ 2213.282400] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 2217.396306] Key type lgssc unregistered [ 2217.956546] LNet: 61051:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2217.974591] LNetError: 61051:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2218.005850] LNet: Removed LNI 192.168.204.141@tcp [ 2219.439203] Key type .llcrypt unregistered [ 2219.445304] Key type ._llcrypt unregistered [ 2261.694193] Key type ._llcrypt registered [ 2261.700588] Key type .llcrypt registered [ 2261.879613] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 2264.235866] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2264.299425] alg: No test for adler32 (adler32-zlib) [ 2265.743569] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 2266.022510] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 2267.751997] Key type lgssc registered [ 2269.221884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2280.466656] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2280.514271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2282.011799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2289.256931] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2301.323191] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2310.795039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2311.251879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2322.885081] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2327.691120] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2327.817900] Lustre: 63893:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2327.833934] Lustre: 63893:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2327.844057] Lustre: 63893:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2327.851536] Lustre: 63893:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2327.858057] Lustre: 63893:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2327.866089] Lustre: 63893:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2335.301829] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2342.745667] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2358.766228] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2358.770205] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2358.821215] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2360.537329] Lustre: server umount lustre-OST0000 complete [ 2366.220214] Lustre: server umount lustre-MDT0000 complete [ 2381.656403] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 2386.518342] Key type lgssc unregistered [ 2386.979580] LNet: 65452:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2386.990934] LNetError: 65452:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2387.032807] LNet: Removed LNI 192.168.204.141@tcp [ 2388.444170] Key type .llcrypt unregistered [ 2388.447091] Key type ._llcrypt unregistered [ 2412.969762] Key type ._llcrypt registered [ 2412.991068] Key type .llcrypt registered [ 2413.336740] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 2436.639334] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 2438.763385] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2438.863661] alg: No test for adler32 (adler32-zlib) [ 2440.242954] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 2440.708061] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 2442.479155] Key type lgssc registered [ 2444.330953] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2492.630816] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2492.676366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2494.218413] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2494.330488] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2494.532860] Lustre: lustre-MDT0000: new disk, initializing [ 2494.922607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2495.036015] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2502.649556] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2519.979018] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2520.157121] Lustre: 68613:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2520.244778] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2520.251817] Lustre: Skipped 1 previous similar message [ 2520.429351] Lustre: lustre-MDT0001: new disk, initializing [ 2520.600221] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2520.661309] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2520.681645] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2527.208489] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2538.991329] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2547.022797] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2556.331570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2556.704352] Lustre: lustre-OST0000: new disk, initializing [ 2556.715433] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2556.727710] Lustre: 70583:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2556.835605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2563.707267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2563.725990] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2563.928092] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2567.760450] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2579.285841] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2585.206918] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2585.462334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2592.708451] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2593.253957] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2599.923289] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2599.927869] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2599.937974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2602.259308] Lustre: server umount lustre-OST0000 complete [ 2610.146451] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2610.160396] Lustre: Skipped 1 previous similar message [ 2610.173755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2610.187473] Lustre: Skipped 1 previous similar message [ 2614.543375] Lustre: server umount lustre-MDT0000 complete [ 2620.027370] LustreError: 68604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785788843 with bad export cookie 14174019394007426183 [ 2620.043259] LustreError: 68604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2620.045718] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2620.500775] Lustre: server umount lustre-MDT0001 complete [ 2631.025335] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:27:31 (1785788851) [ 2643.761240] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 2695.245378] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 2709.632654] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 2753.515457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2753.864152] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2753.923951] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2754.061863] Lustre: lustre-MDT0000: new disk, initializing [ 2754.230521] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2754.276146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2762.215302] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2777.596863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2777.778932] Lustre: 76612:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2777.859992] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2777.871592] Lustre: Skipped 1 previous similar message [ 2778.005780] Lustre: lustre-MDT0001: new disk, initializing [ 2778.102184] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2778.162130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2778.181745] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2786.881341] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2801.058463] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2810.773632] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2819.438786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2819.824500] Lustre: lustre-OST0000: new disk, initializing [ 2819.832403] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2819.847500] Lustre: 78583:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2819.976188] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2828.766191] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2828.899598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2828.913536] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2829.063933] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2840.478141] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2845.744736] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2845.924909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2851.457685] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2851.814772] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2854.881680] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2854.893685] LustreError: Skipped 1 previous similar message [ 2854.896594] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2854.902360] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2854.931739] Lustre: Skipped 2 previous similar messages [ 2860.038415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2860.056297] Lustre: Skipped 2 previous similar messages [ 2861.219597] Lustre: server umount lustre-OST0000 complete [ 2870.245357] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2870.255212] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2870.258346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2872.791224] Lustre: server umount lustre-MDT0000 complete [ 2875.360732] LustreError: 76623:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2875.374859] LustreError: 76623:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2877.224818] LustreError: 80125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785789100 with bad export cookie 14174019394007427065 [ 2877.234853] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2877.243018] LustreError: 80125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2877.973084] Lustre: server umount lustre-MDT0001 complete [ 2887.878172] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:31:48 (1785789108) [ 2896.350658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2897.026481] LustreError: 80997:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2897.117957] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2902.510375] LustreError: 80998:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2904.082461] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2906.595447] LustreError: 80997:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2911.717031] LustreError: 80998:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2915.351163] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2921.759336] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2932.090141] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2938.892550] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2947.549602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2948.229765] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2948.243385] Lustre: Skipped 1 previous similar message [ 2957.115037] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 2965.855949] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2969.969746] Lustre: 83671:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 2b9ada10-f1d4-4f10-ad81-229412274c38 at adminstrative request [ 2972.135858] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2972.153707] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2982.372891] Lustre: MGS: Client 2b9ada10-f1d4-4f10-ad81-229412274c38 (at 0@lo) reconnecting [ 2982.376684] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xc4b43b8aab4f6b6b to 0xc4b43b8aab4f6e0b [ 2982.404986] Lustre: MGC192.168.204.141@tcp: Connection restored to 0@lo (at 0@lo) [ 3008.774842] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3009.047449] Lustre: lustre-OST0001: new disk, initializing [ 3009.063384] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3009.073513] Lustre: 84062:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3015.290820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3015.311721] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3015.481506] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3017.790947] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3029.557161] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3036.140082] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3036.142698] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3036.153415] LustreError: Skipped 1 previous similar message [ 3036.155355] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3036.155360] Lustre: Skipped 1 previous similar message [ 3036.182235] Lustre: Skipped 2 previous similar messages [ 3039.582743] Lustre: server umount lustre-OST0001 complete [ 3041.271161] LustreError: 84528:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3041.288178] LustreError: 84528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3043.811821] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3044.833032] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3044.857637] Lustre: Skipped 1 previous similar message [ 3056.619150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3056.629675] Lustre: Skipped 7 previous similar messages [ 3058.144021] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3058.650475] Lustre: server umount lustre-OST0000 complete [ 3064.289932] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3064.305303] LustreError: Skipped 1 previous similar message [ 3064.311522] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3070.079286] Lustre: server umount lustre-MDT0000 complete [ 3071.970609] LustreError: 81725:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3071.990790] LustreError: 81725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3074.429860] LustreError: 80979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785789297 with bad export cookie 14174019394007428619 [ 3074.435501] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3074.442339] LustreError: 80979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3075.157714] Lustre: server umount lustre-MDT0001 complete [ 3084.455201] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:35:05 (1785789305) [ 3086.136426] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3088.192669] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:35:09 (1785789309) [ 3090.309661] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3092.170815] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:35:13 (1785789313) [ 3093.495267] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3095.198378] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:35:16 (1785789316) [ 3097.239755] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3099.365539] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:35:20 (1785789320) [ 3101.677365] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3108.676252] loop: module loaded [ 3108.734099] loop0: detected capacity change from 0 to 4294967296 [ 3108.818740] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3110.988037] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3116.257587] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3121.690197] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3155.331043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3155.387230] LustreError: 88195:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3155.403149] LustreError: 88195:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3155.421040] LustreError: 88195:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3155.431852] LustreError: 88195:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3181.861959] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 3194.374592] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 3238.196054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3238.632695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3238.720860] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3238.866272] Lustre: lustre-MDT0000: new disk, initializing [ 3238.999594] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3239.007548] Lustre: Skipped 1 previous similar message [ 3239.066184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3243.814386] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3256.706818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3256.801694] Lustre: 91061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3257.094712] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3262.354638] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3275.684923] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3283.435941] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3292.403738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3292.763330] Lustre: 93032:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3300.329563] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3303.003365] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3303.133370] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3310.737034] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3316.751900] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3317.166694] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3322.626736] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3322.936664] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3329.022461] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3329.042025] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3329.044229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3329.044235] Lustre: Skipped 3 previous similar messages [ 3329.050728] Lustre: Skipped 2 previous similar messages [ 3329.084223] LustreError: Skipped 1 previous similar message [ 3331.904430] Lustre: server umount lustre-OST0000 complete [ 3343.307329] Lustre: server umount lustre-MDT0000 complete [ 3344.353460] LustreError: 91074:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3344.369849] LustreError: 91074:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3347.766270] LustreError: 91054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785789571 with bad export cookie 14174019394007429235 [ 3347.775666] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3347.784599] LustreError: 91054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3348.507774] Lustre: server umount lustre-MDT0001 complete [ 3359.153119] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:39:39 (1785789579) [ 3366.744871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3367.464743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3367.475580] Lustre: Skipped 2 previous similar messages [ 3373.263503] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3383.567234] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3389.532895] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3399.305588] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3410.603955] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3421.434449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3433.203451] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3445.406062] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3451.823454] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3452.159267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3458.390257] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3458.751059] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3466.231091] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3466.459337] Lustre: lustre-OST0001: new disk, initializing [ 3466.463028] Lustre: Skipped 2 previous similar messages [ 3466.474025] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3466.477586] Lustre: Skipped 3 previous similar messages [ 3466.489882] Lustre: 98829:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3473.168376] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3475.102583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3475.119020] Lustre: Skipped 2 previous similar messages [ 3475.127473] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3475.296811] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3484.418199] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3491.057903] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3491.353263] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3497.000474] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3497.315966] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3521.590850] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3551.307463] Lustre: Failing over lustre-MDT0000 [ 3551.741794] Lustre: server umount lustre-MDT0000 complete [ 3552.224401] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3552.229474] LustreError: 95445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3552.247735] Lustre: Skipped 5 previous similar messages [ 3552.268474] LustreError: 95445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3567.584169] Lustre: 66831:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785789775/real 1785789775] req@0000000020a66bec x1872535134446592/t0(0) o400->MGC192.168.204.141@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785789791 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3567.631434] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3578.506202] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3578.508895] LDISKFS-fs (dm-0): recovery complete [ 3578.520622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3593.205597] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xc4b43b8aab5038b7 [ 3593.223145] Lustre: MGC192.168.204.141@tcp: Connection restored to 0@lo (at 0@lo) [ 3593.572331] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3596.865847] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3596.867184] Lustre: 95440:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3598.834323] Lustre: 96183:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3598.848527] Lustre: 96183:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 3598.867168] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 3599.010993] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3599.345980] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3599.684614] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3599.702504] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3600.018642] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3600.048065] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3600.061605] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3600.083466] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3600.105398] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 3600.757893] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3600.777326] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3601.111092] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3601.125830] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3602.154423] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3602.165491] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 3602.819038] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3602.826758] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3603.151120] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3603.169622] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3606.391548] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3606.408873] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 3607.060595] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3607.082598] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3607.415671] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3607.432282] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3614.506089] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3614.524601] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 3615.200236] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3615.227314] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3615.551077] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3615.560856] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3630.514151] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3630.530336] Lustre: 101684:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 49 previous similar messages [ 3631.498665] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3631.511416] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3631.839160] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3631.851347] LustreError: 101684:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3656.071225] Lustre: 101684:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3656.082846] Lustre: 101684:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3656.102375] Lustre: 101684:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client f511a8e5-ca09-495a-a0d1-ae6a933b2baf@192.168.204.41@tcp [ 3656.136783] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3656.154595] Lustre: 101684:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@000000001372b818 x1872535106952064/t0(8589934778) o35->f511a8e5-ca09-495a-a0d1-ae6a933b2baf@192.168.204.41@tcp:470/0 lens 392/0 e 0 to 0 dl 1785789890 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3656.199084] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3656.219709] Lustre: lustre-MDT0000: Denying connection for new client f511a8e5-ca09-495a-a0d1-ae6a933b2baf (at 192.168.204.41@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3656.242277] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3656.316149] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3656.316661] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3656.340359] Lustre: Skipped 2 previous similar messages [ 3656.418084] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3656.422028] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3675.107775] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3675.122608] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3675.129484] Lustre: Skipped 3 previous similar messages [ 3679.708694] Lustre: server umount lustre-OST0000 complete [ 3680.740207] LustreError: 97281:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3680.767804] LustreError: 97281:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 51 previous similar messages [ 3685.353091] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3685.370509] Lustre: Skipped 2 previous similar messages [ 3699.170319] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3699.574225] Lustre: server umount lustre-OST0001 complete [ 3710.547651] Lustre: server umount lustre-MDT0000 complete [ 3714.735857] LustreError: 95425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785789938 with bad export cookie 14174019394007480503 [ 3714.737112] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3714.746922] LustreError: 95425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3715.392979] Lustre: server umount lustre-MDT0001 complete [ 3725.416129] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:45:46 (1785789946) [ 3734.131640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3734.142198] Lustre: *** cfs_fail_loc=197, val=0*** [ 3749.727600] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3754.847489] LustreError: 103661:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.141@tcp: failed processing log, type 4: rc = -110 [ 3759.996631] Lustre: *** cfs_fail_loc=197, val=0*** [ 3760.003394] Lustre: Skipped 65 previous similar messages [ 3780.575516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3780.586803] Lustre: Skipped 4 previous similar messages [ 3787.591179] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3795.946887] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3798.479853] Lustre: server umount lustre-OST0000 complete [ 3808.374476] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:47:09 (1785790029) [ 3830.137363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3836.515693] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3848.423478] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3852.956610] LustreError: 105311:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3852.976919] LustreError: 105311:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3852.986143] LustreError: 105311:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3853.640102] Lustre: server umount lustre-MDT0000 complete [ 3892.137924] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 3905.161943] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 3949.732950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3950.142554] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3950.201184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3950.340727] Lustre: lustre-MDT0000: new disk, initializing [ 3950.508973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3956.112268] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3968.889484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3969.022561] Lustre: 109491:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3969.472846] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3975.340763] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 3986.168537] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3993.354673] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4001.586115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4001.900510] Lustre: 111460:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4009.414263] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4009.708739] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4021.977353] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4028.144369] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4028.479496] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4033.742337] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4034.019543] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4040.673629] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4040.683223] LustreError: Skipped 1 previous similar message [ 4040.688337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4040.698944] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4040.708333] Lustre: Skipped 4 previous similar messages [ 4040.725185] Lustre: Skipped 10 previous similar messages [ 4042.618210] Lustre: server umount lustre-OST0000 complete [ 4056.535690] Lustre: server umount lustre-MDT0000 complete [ 4061.189921] LustreError: 109502:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4061.269853] LustreError: 109502:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [ 4064.851209] LustreError: 110285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785790288 with bad export cookie 14174019394007491059 [ 4064.861979] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4064.864101] LustreError: 110285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4065.386581] Lustre: server umount lustre-MDT0001 complete [ 4083.935115] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:51:44 (1785790304) [ 4097.177097] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 4101.441481] Key type lgssc unregistered [ 4101.929611] LNet: 113951:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4101.945885] LNetError: 113951:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4101.969647] LNet: Removed LNI 192.168.204.141@tcp [ 4103.339920] Key type .llcrypt unregistered [ 4103.341621] Key type ._llcrypt unregistered [ 4131.275550] Key type ._llcrypt registered [ 4131.288085] Key type .llcrypt registered [ 4131.804206] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 4158.107172] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 4159.670307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4159.759747] alg: No test for adler32 (adler32-zlib) [ 4161.290249] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 4161.636828] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 4163.504596] Key type lgssc registered [ 4166.138057] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4229.172206] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4229.187904] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4230.449797] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4230.475358] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4230.576777] Lustre: lustre-MDT0000: new disk, initializing [ 4230.651354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4230.665558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4235.491769] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4247.283623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4247.641816] Lustre: lustre-OST0000: new disk, initializing [ 4247.649323] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4247.659465] Lustre: Skipped 1 previous similar message [ 4247.674672] Lustre: 118131:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4247.789514] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4257.058819] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4257.401122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4257.413611] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4257.439924] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4271.820900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4272.028615] Lustre: lustre-OST0001: new disk, initializing [ 4272.032310] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4272.050945] Lustre: 118994:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4272.135716] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4278.428311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4278.447101] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4278.498777] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4280.212880] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4293.602157] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4293.617254] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4293.630091] Lustre: Skipped 1 previous similar message [ 4293.646704] Lustre: Skipped 1 previous similar message [ 4297.565817] Lustre: server umount lustre-MDT0000 complete [ 4305.730799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4311.535193] LustreError: 119520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785790534 with bad export cookie 9533893792186677309 [ 4311.540941] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4311.805913] Lustre: server umount lustre-OST0000 complete [ 4316.945917] Lustre: server umount lustre-OST0001 complete [ 4328.485396] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 16:55:48 (1785790548) [ 4364.533931] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 16:56:23 (1785790583) [ 4385.104383] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 4413.583925] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:57:14 (1785790634) [ 4462.405222] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 4478.336414] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 4527.656296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4527.995852] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4528.028342] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4528.260040] Lustre: lustre-MDT0000: new disk, initializing [ 4528.532078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4528.569141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4535.719856] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4549.306560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4549.507232] Lustre: 126798:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4549.649433] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4549.658634] Lustre: Skipped 1 previous similar message [ 4549.835545] Lustre: lustre-MDT0001: new disk, initializing [ 4549.995783] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4550.071879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4550.098740] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4555.616953] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4567.262742] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4575.091966] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4583.665079] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4583.999332] Lustre: lustre-OST0000: new disk, initializing [ 4584.003707] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4584.015496] Lustre: 128774:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4584.130853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4589.189808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4589.203972] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4589.374665] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4593.033275] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4603.782342] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4609.968781] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4610.283279] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4616.824175] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4617.255167] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4620.263884] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4620.271740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4620.289781] Lustre: Skipped 1 previous similar message [ 4620.295823] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4620.300947] Lustre: Skipped 1 previous similar message [ 4625.383773] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4625.394997] Lustre: Skipped 1 previous similar message [ 4626.047819] Lustre: server umount lustre-OST0000 complete [ 4632.032203] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4632.049265] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4632.075556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4635.623627] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4637.918491] Lustre: server umount lustre-MDT0000 complete [ 4643.332585] LustreError: 126792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785790866 with bad export cookie 9533893792186678534 [ 4643.345720] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4643.346681] LustreError: 126792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4644.656581] Lustre: server umount lustre-MDT0001 complete [ 4660.382151] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:01:19 (1785790879) [ 4675.301412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4676.290670] LustreError: 131200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4676.334017] LustreError: 131200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4676.403360] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4681.697097] LustreError: 131201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4682.923244] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4685.795242] LustreError: 131200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4690.915739] LustreError: 131201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4693.959374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4701.961486] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4712.169183] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4718.643429] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4730.026912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4731.191508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4731.216838] Lustre: Skipped 1 previous similar message [ 4745.508707] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4756.103935] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4761.937061] Lustre: *** cfs_fail_loc=159, val=0*** [ 4761.948876] Lustre: Skipped 1 previous similar message [ 4777.581097] Lustre: lustre-MDT0000: Client 928633d6-907f-46bd-826f-e1db8c468c1e (at 192.168.204.41@tcp) reconnecting [ 4779.042069] Lustre: *** cfs_fail_loc=159, val=0*** [ 4779.062058] Lustre: Skipped 5 previous similar messages [ 4794.475730] Lustre: lustre-MDT0000: Client 928633d6-907f-46bd-826f-e1db8c468c1e (at 192.168.204.41@tcp) reconnecting [ 4800.996896] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4801.003866] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4801.027462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4801.029470] Lustre: Skipped 2 previous similar messages [ 4803.049342] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4813.292719] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4813.309096] Lustre: Skipped 5 previous similar messages [ 4815.327696] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4815.713410] Lustre: server umount lustre-OST0000 complete [ 4822.500019] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4822.502690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4822.521799] Lustre: Skipped 1 previous similar message [ 4827.155568] Lustre: server umount lustre-MDT0000 complete [ 4828.640623] LustreError: 131201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4828.652976] LustreError: 131201:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4831.782957] LustreError: 133920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785791055 with bad export cookie 9533893792186679416 [ 4831.785878] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4831.795173] LustreError: 133920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4832.443765] Lustre: server umount lustre-MDT0001 complete [ 4847.244845] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 4851.218179] Key type lgssc unregistered [ 4851.597470] LNet: 134979:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4851.623098] LNetError: 134979:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4851.641657] LNet: Removed LNI 192.168.204.141@tcp [ 4853.017396] Key type .llcrypt unregistered [ 4853.023174] Key type ._llcrypt unregistered [ 4884.265245] Key type ._llcrypt registered [ 4884.271292] Key type .llcrypt registered [ 4884.448326] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 4886.026272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4886.089825] alg: No test for adler32 (adler32-zlib) [ 4887.376952] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 4887.765693] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 4889.487280] Key type lgssc registered [ 4891.378091] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4903.372464] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4903.401815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4904.915656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4910.957503] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4915.170750] LustreError: 136395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4915.215420] LustreError: 136395:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4920.310242] LustreError: 136394:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4922.502253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4923.187210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4930.002351] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4939.276421] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4950.340016] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4966.753088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4968.119434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4975.352662] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4979.868691] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 4992.643604] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5005.401141] Lustre: *** cfs_fail_loc=159, val=0*** [ 5021.861224] Lustre: lustre-MDT0000: Client 464f5f58-7913-497c-b101-e66eb49c1641 (at 192.168.204.41@tcp) reconnecting [ 5025.642110] Lustre: *** cfs_fail_loc=159, val=0*** [ 5042.316377] Lustre: lustre-MDT0001: Client 464f5f58-7913-497c-b101-e66eb49c1641 (at 192.168.204.41@tcp) reconnecting [ 5044.633243] Lustre: *** cfs_fail_loc=159, val=0*** [ 5044.634722] Lustre: Skipped 3 previous similar messages [ 5061.265256] Lustre: lustre-MDT0001: Client 464f5f58-7913-497c-b101-e66eb49c1641 (at 192.168.204.41@tcp) reconnecting [ 5077.110097] Lustre: 139462:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5080.338066] Lustre: *** cfs_fail_loc=159, val=0*** [ 5080.349276] Lustre: Skipped 4 previous similar messages [ 5095.518712] Lustre: lustre-MDT0000: Client 75c449bb-fcf3-4b74-881d-7916e7c28ab5 (at 192.168.204.41@tcp) reconnecting [ 5097.801730] Lustre: *** cfs_fail_loc=159, val=0*** [ 5097.803373] Lustre: Skipped 14 previous similar messages [ 5113.486742] Lustre: lustre-MDT0000: Client 75c449bb-fcf3-4b74-881d-7916e7c28ab5 (at 192.168.204.41@tcp) reconnecting [ 5115.157984] Lustre: 139754:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5123.040974] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5123.048940] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5123.064509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5124.066783] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5124.077831] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5124.080867] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5129.177053] Lustre: server umount lustre-OST0000 complete [ 5134.308271] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5134.311681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5134.331603] Lustre: Skipped 1 previous similar message [ 5134.347891] Lustre: Skipped 2 previous similar messages [ 5136.651934] Lustre: server umount lustre-MDT0000 complete [ 5139.424929] LustreError: 139583:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5139.465967] LustreError: 139583:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5142.852540] LustreError: 136377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785791366 with bad export cookie 6509380744681130590 [ 5142.874011] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5142.885144] LustreError: 136377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5143.713906] Lustre: server umount lustre-MDT0001 complete [ 5158.448542] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 5162.865974] Key type lgssc unregistered [ 5163.274600] LNet: 140811:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5163.281054] LNetError: 140811:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5163.323136] LNet: Removed LNI 192.168.204.141@tcp [ 5164.601761] Key type .llcrypt unregistered [ 5164.605063] Key type ._llcrypt unregistered [ 5194.702461] Key type ._llcrypt registered [ 5194.706450] Key type .llcrypt registered [ 5194.806455] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 5195.839071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5195.986671] alg: No test for adler32 (adler32-zlib) [ 5197.421388] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 5197.793154] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 5199.471179] Key type lgssc registered [ 5200.700826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5209.651212] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5209.682405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5211.291474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5217.511185] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5220.838285] LustreError: 142230:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5220.860365] LustreError: 142230:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5225.959132] LustreError: 142231:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5228.117821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5228.546694] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5236.062307] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5247.912454] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5258.111575] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5269.937335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5270.452422] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5273.535161] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5279.099411] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5280.708942] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5290.096310] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5297.358468] Lustre: 144816:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5304.183524] Lustre: 144865:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5306.301410] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5307.887135] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5315.265635] Lustre: 145020:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5316.679164] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5318.708328] Lustre: 145121:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5326.831898] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5326.887744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5326.892304] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5331.952851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5331.957893] Lustre: Skipped 2 previous similar messages [ 5332.734263] Lustre: server umount lustre-OST0000 complete [ 5341.160886] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5341.170777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5341.184543] Lustre: Skipped 1 previous similar message [ 5341.190352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5342.205023] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5344.260713] Lustre: server umount lustre-MDT0000 complete [ 5347.297970] LustreError: 143667:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5347.321969] LustreError: 143667:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5348.763387] LustreError: 144915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785791572 with bad export cookie 9074006593297305866 [ 5348.771994] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5348.773905] LustreError: 144915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5349.276865] Lustre: server umount lustre-MDT0001 complete [ 5364.111530] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 5367.620368] Key type lgssc unregistered [ 5367.885737] LNet: 146181:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5367.898259] LNetError: 146181:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5367.927605] LNet: Removed LNI 192.168.204.141@tcp [ 5369.292483] Key type .llcrypt unregistered [ 5369.296609] Key type ._llcrypt unregistered [ 5400.959287] Key type ._llcrypt registered [ 5400.962708] Key type .llcrypt registered [ 5401.182307] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 5403.138730] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5403.173251] alg: No test for adler32 (adler32-zlib) [ 5404.172308] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 5404.391918] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 5406.063750] Key type lgssc registered [ 5407.768815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5419.251065] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5419.282885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5421.170554] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5425.840119] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5430.756429] LustreError: 147600:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5430.777644] LustreError: 147600:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5435.875751] LustreError: 147599:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5438.695428] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5439.392620] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5444.111239] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5454.049398] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5460.060182] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5468.225510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5468.719517] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5476.991970] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5484.083417] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5486.417268] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5491.261190] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5496.875180] Lustre: *** cfs_fail_loc=159, val=0*** [ 5513.374515] Lustre: lustre-MDT0000: Client 5ee21360-5efe-4787-a476-8875ef4faac0 (at 192.168.204.41@tcp) reconnecting [ 5521.888279] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5521.894063] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5521.924073] Lustre: Skipped 1 previous similar message [ 5527.019171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5527.042277] Lustre: Skipped 2 previous similar messages [ 5527.283945] Lustre: server umount lustre-OST0000 complete [ 5536.738844] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5536.757812] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5536.790347] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5539.202298] Lustre: server umount lustre-MDT0000 complete [ 5542.371259] LustreError: 148611:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5542.395330] LustreError: 148611:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5543.945310] LustreError: 147581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785791767 with bad export cookie 12071786048676372295 [ 5543.954257] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5543.974479] LustreError: 147581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5544.736252] Lustre: server umount lustre-MDT0001 complete [ 5561.898583] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 5569.106937] Key type lgssc unregistered [ 5569.876825] LNet: 151265:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5569.898058] LNetError: 151265:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5570.923924] LNet: Removed LNI 192.168.204.141@tcp [ 5572.519815] Key type .llcrypt unregistered [ 5572.521068] Key type ._llcrypt unregistered [ 5611.119537] Key type ._llcrypt registered [ 5611.122434] Key type .llcrypt registered [ 5611.226649] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 5612.281568] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5612.416092] alg: No test for adler32 (adler32-zlib) [ 5613.531141] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 5614.349657] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 5616.192747] Key type lgssc registered [ 5619.746431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5637.757392] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5637.828617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5640.154986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5650.405379] LustreError: 152684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5650.429108] LustreError: 152684:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5651.159366] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5655.521145] LustreError: 152685:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5660.642472] LustreError: 152684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5664.552690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5664.738949] LustreError: 152685:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5665.111260] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5672.134702] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5683.344581] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5691.485953] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5701.361520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5702.033810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5710.304541] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5710.338571] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5712.674038] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5722.726444] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5744.727321] Lustre: lustre-MDT0001: evicting 8b535944-fce4-4e05-b8df-639f9d97c578 (at 192.168.204.41@tcp) by administrative request [ 5744.731821] Lustre: Skipped 1 previous similar message [ 5776.867020] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5776.876253] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5776.881505] LustreError: Skipped 1 previous similar message [ 5776.890697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5776.911529] Lustre: Skipped 1 previous similar message [ 5781.988451] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5782.017798] Lustre: Skipped 2 previous similar messages [ 5787.106956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5787.124866] Lustre: Skipped 1 previous similar message [ 5789.152110] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5789.409259] Lustre: server umount lustre-OST0000 complete [ 5797.349536] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5797.351164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5797.379447] Lustre: Skipped 1 previous similar message [ 5797.401705] Lustre: Skipped 1 previous similar message [ 5802.050406] Lustre: server umount lustre-MDT0000 complete [ 5802.465642] LustreError: 153616:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5802.491992] LustreError: 153616:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5806.747888] LustreError: 152664:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785792030 with bad export cookie 3110774555482958753 [ 5806.771052] LustreError: 152664:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5806.772284] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5807.724657] Lustre: server umount lustre-MDT0001 complete [ 5822.553857] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 5826.881285] Key type lgssc unregistered [ 5827.242756] LNet: 156765:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5827.260139] LNetError: 156765:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5827.278987] LNet: Removed LNI 192.168.204.141@tcp [ 5828.567909] Key type .llcrypt unregistered [ 5828.572159] Key type ._llcrypt unregistered [ 5938.595181] Key type ._llcrypt registered [ 5938.602376] Key type .llcrypt registered [ 5938.853488] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 5941.991136] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5942.067708] alg: No test for adler32 (adler32-zlib) [ 5943.613747] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 5944.109169] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 5945.775163] Key type lgssc registered [ 5948.212539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5962.834253] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5962.868792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5964.938278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5972.095188] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 5975.530169] LustreError: 159261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5975.545946] LustreError: 159261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5980.640873] LustreError: 159262:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5984.079820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5984.935265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5991.666389] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6004.675628] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6014.639196] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6024.442475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6024.775304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6029.877348] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 6033.218936] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6034.985284] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 6044.559278] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6060.521740] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6060.537985] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6060.545447] Lustre: Skipped 1 previous similar message [ 6060.556403] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6063.162686] Lustre: server umount lustre-OST0000 complete [ 6070.761223] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6070.766551] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6070.779203] Lustre: Skipped 1 previous similar message [ 6070.795178] Lustre: Skipped 1 previous similar message [ 6075.879279] LustreError: 159256:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6075.893170] LustreError: 159256:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6076.051977] Lustre: server umount lustre-MDT0000 complete [ 6080.999441] LustreError: 159257:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6081.031046] LustreError: 159257:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6081.232249] LustreError: 159242:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785792304 with bad export cookie 2707686422318278139 [ 6081.233345] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6081.254901] LustreError: 159242:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6081.901270] Lustre: server umount lustre-MDT0001 complete [ 6097.507461] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 6101.944046] Key type lgssc unregistered [ 6102.444261] LNet: 162809:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6102.469835] LNetError: 162809:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6103.526202] LNet: Removed LNI 192.168.204.141@tcp [ 6104.922094] Key type .llcrypt unregistered [ 6104.925641] Key type ._llcrypt unregistered [ 6143.688861] Key type ._llcrypt registered [ 6143.691199] Key type .llcrypt registered [ 6143.781632] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 6145.012981] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6145.076127] alg: No test for adler32 (adler32-zlib) [ 6146.966413] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 6147.430945] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 6149.320870] Key type lgssc registered [ 6151.531554] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6178.515870] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:26:38 (1785792398) [ 6205.617219] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [ 6222.727934] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 6269.972753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6270.028220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6271.454702] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6271.564523] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6271.745576] Lustre: lustre-MDT0000: new disk, initializing [ 6271.940714] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6271.989134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6277.995317] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6290.684041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6290.844268] Lustre: 167100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6290.999893] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6291.011270] Lustre: Skipped 1 previous similar message [ 6291.149328] Lustre: lustre-MDT0001: new disk, initializing [ 6291.366741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6291.459604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6291.478234] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6297.980966] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6312.885414] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6321.564837] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6331.190382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6331.826752] Lustre: lustre-OST0000: new disk, initializing [ 6331.839899] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6331.850457] Lustre: 169069:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6332.062794] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6341.317169] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6342.256313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6342.289503] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6342.454109] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6355.431513] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6373.345463] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6373.348800] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6373.367696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6376.085513] Lustre: server umount lustre-OST0000 complete [ 6383.584726] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6383.584777] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6383.608515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6383.625562] Lustre: Skipped 2 previous similar messages [ 6383.654260] Lustre: Skipped 2 previous similar messages [ 6388.141883] Lustre: server umount lustre-MDT0000 complete [ 6392.915897] LustreError: 167094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785792616 with bad export cookie 15299957778104473823 [ 6392.925680] LustreError: 167094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6392.926732] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6393.673775] Lustre: server umount lustre-MDT0001 complete [ 6409.294143] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 6412.492555] Key type lgssc unregistered [ 6413.009590] LNet: 170978:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6413.023539] LNetError: 170978:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6413.041057] LNet: Removed LNI 192.168.204.141@tcp [ 6414.191782] Key type .llcrypt unregistered [ 6414.195593] Key type ._llcrypt unregistered [ 6448.379845] Key type ._llcrypt registered [ 6448.382649] Key type .llcrypt registered [ 6448.511148] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 6449.848454] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6450.057566] alg: No test for adler32 (adler32-zlib) [ 6451.319608] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 6451.615805] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 6453.311672] Key type lgssc registered [ 6454.594611] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6463.989950] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6464.022638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6465.910574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6473.530342] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6476.258711] LustreError: 172394:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6476.280233] LustreError: 172394:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6481.377395] LustreError: 172395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6485.000031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6486.026370] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6495.527596] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6510.324642] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6519.784386] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6534.266136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6534.769349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6535.843085] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6546.650272] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 6561.563385] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6569.309052] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6569.334617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6569.363871] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6569.375887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.393165] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6570.822419] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6570.867728] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6570.888319] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6570.898988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.925608] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6572.071356] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6572.112924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6572.129475] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6572.139748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.151955] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6573.323063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.661362] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6574.666766] Lustre: Skipped 1 previous similar message [ 6574.679925] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6574.686760] Lustre: Skipped 1 previous similar message [ 6574.690069] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6574.695256] Lustre: Skipped 1 previous similar message [ 6574.698381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.705928] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6574.708258] Lustre: Skipped 1 previous similar message [ 6575.815298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.220807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.367778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.632048] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6579.637391] Lustre: Skipped 3 previous similar messages [ 6579.656305] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6579.676809] Lustre: Skipped 3 previous similar messages [ 6579.688816] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6579.699790] Lustre: Skipped 3 previous similar messages [ 6579.713148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.734436] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6579.745973] Lustre: Skipped 3 previous similar messages [ 6580.784568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.964154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.079228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.266799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.595662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.010630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.270614] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6588.280556] Lustre: Skipped 6 previous similar messages [ 6588.304668] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6588.324679] Lustre: Skipped 6 previous similar messages [ 6588.334795] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6588.340392] Lustre: Skipped 6 previous similar messages [ 6588.348416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.364509] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6588.380733] Lustre: Skipped 6 previous similar messages [ 6589.402251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.602338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.929748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.444194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.809984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.867368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.033459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.481892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.657599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.887839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.410616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.756695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.022273] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6605.040427] Lustre: Skipped 12 previous similar messages [ 6605.066816] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6605.089478] Lustre: Skipped 12 previous similar messages [ 6605.111544] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6605.122910] Lustre: Skipped 12 previous similar messages [ 6605.131644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.180215] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6605.199608] Lustre: Skipped 12 previous similar messages [ 6606.492210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.009071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.431459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.820725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.894283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.323572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.628048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.971356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.686099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.293573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.161368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.392520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.630641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.981263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.117635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.369579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.879916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.186712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.774507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.357729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.693190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.068550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.349373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.051902] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6638.057717] Lustre: Skipped 23 previous similar messages [ 6638.091922] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6638.101293] Lustre: Skipped 23 previous similar messages [ 6638.106245] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6638.111472] Lustre: Skipped 23 previous similar messages [ 6638.114365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.125154] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6638.142393] Lustre: Skipped 23 previous similar messages [ 6639.559319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.823577] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6640.847744] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6640.896247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.439202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.312911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.793448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.105352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.343238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.591832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.093054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.793446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.126996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.411067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.315751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.143016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.810645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.679337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.501173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.919634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.302941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.133911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.247393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.742499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.099373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.034651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.377755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.892725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.120924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.637481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.108848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.385972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.930508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.433706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.290772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.731650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.188997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.711285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.974463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.281540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.473228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.690063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.039848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.037935] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6703.046932] Lustre: Skipped 41 previous similar messages [ 6703.087976] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6703.106566] Lustre: Skipped 41 previous similar messages [ 6703.112285] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6703.120135] Lustre: Skipped 41 previous similar messages [ 6703.126643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.139901] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6703.152749] Lustre: Skipped 41 previous similar messages [ 6704.707409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.279834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.651123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.129160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.418509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.093651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.430239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.840648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.205054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.374319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.936928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.580540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.868217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.575805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.089407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.584215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.956412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.348974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.618376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.932671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.139393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.487110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.837873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.168564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.685057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.151661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.641286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.103926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.518452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.729531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.007910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.366635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.492678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.755910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.240252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.542906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.778440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.994978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.276794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.606661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.946276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.166985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.643575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.852581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.116741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.269761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.537488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.900857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.997954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.047592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.087928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.412952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.783032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.101803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.357600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.714357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.037234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.312914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.575239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.010435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.918462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.971883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.962217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.030487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.921898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.180953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.474176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.887994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.301437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.760114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.047626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.644860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.025694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.172121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.548360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.008076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.422909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.904404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.531568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.920458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.470320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.614092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.979964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.646525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.263285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.808779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.070043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.895365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.245749] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6823.256929] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6823.288541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.577767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.238257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.186992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.893754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.568923] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6831.592708] Lustre: Skipped 93 previous similar messages [ 6831.634564] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6831.646264] Lustre: Skipped 93 previous similar messages [ 6831.666235] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6831.671575] Lustre: Skipped 93 previous similar messages [ 6831.702642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.735457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6831.758941] Lustre: Skipped 93 previous similar messages [ 6833.601091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.788660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.394705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6839.214823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.491399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.825029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.931590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.873407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.601028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.736063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.606519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.995871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.301845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.521418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.334750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.183619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.015742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.633505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.283430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.524417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.745845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.881479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.166390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.424529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.684627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.947184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.287485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.617320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.944871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.144278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.678200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.053592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.343480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.768310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.089889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.413364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.651831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.823475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.944191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.048616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.306263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.474173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.479636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.573104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.565160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.759900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.864755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.843568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.285820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.607573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.759660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.966666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.180561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6907.378041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.519695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6909.622546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.755286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.948571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6913.034370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6913.933605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.903551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6916.131414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.449124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.822546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.202503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.622276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.898964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.073475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6925.380376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.621118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.806364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.007636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.291405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6931.203614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.508903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.837479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.029727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.339335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.759193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.110507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.510784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6941.844241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6943.480738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.917725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6946.302416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6947.630246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6949.016114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6950.290908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6951.726053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6953.120711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6954.498245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6955.889906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.211732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.466482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6959.904893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6960.971804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6962.083927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6963.092351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.410066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6965.954804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6967.250859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6968.625936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6969.984954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6971.281366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.627738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.088579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.393788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6976.741030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6978.052276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6979.461620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.953257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6982.330097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.919756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.268581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6986.757282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6987.994197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6989.047408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.281135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6991.719317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.087404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6994.437404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6995.734352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6997.017411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.239538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6999.546666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7001.024678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.481950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7003.781348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7004.969175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7006.245540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7007.622863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7009.149781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7010.312970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.480247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7012.642722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7013.976993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7015.160160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.393382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7017.669427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7018.854595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7020.090985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.436035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7022.656880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7023.972180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7025.617525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7027.300909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7028.483599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.854402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7031.301831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7032.469295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7033.410490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7034.872395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7036.560936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7037.249721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7038.322493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.703418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7041.167102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7042.219955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7043.708542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7044.791376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7046.139303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7047.295735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7048.989576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7050.252996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7051.969039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7053.599779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7055.253563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7056.751142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7058.265178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.151234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7062.025653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7063.929265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7065.588319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7067.291264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7068.694950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7070.099575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7071.470917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.407878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7073.818671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.046358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.997385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7077.082062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.293533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.465941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.615298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7081.789297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7082.651146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7083.606688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7084.778524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7085.835730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.163976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7088.516156] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7088.559503] Lustre: Skipped 191 previous similar messages [ 7088.569382] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7088.583371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7088.613770] Lustre: Skipped 191 previous similar messages [ 7088.619645] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7088.626204] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7088.644578] Lustre: Skipped 191 previous similar messages [ 7088.656523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7088.694897] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7088.707921] Lustre: Skipped 191 previous similar messages [ 7089.951976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.362057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7092.335820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7093.426668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7094.461213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7095.774158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.862474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7097.803933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7098.995716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7100.115350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7101.428893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7102.651051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7103.812131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7105.009049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.303537] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7106.315612] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7106.319426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7107.442603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7108.722111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7110.024791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7111.371881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7112.723168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7113.850419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7115.080299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7116.314835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7117.296555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.615867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.065444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7121.538293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7122.847218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.719895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.393786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7125.229805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7126.320580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.703933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.910461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7130.304735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7132.133632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7133.635364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.152911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.144152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.417437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.787037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7142.978130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.986808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7147.000484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7149.360542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7151.314864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.503076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7155.447944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.530757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7159.226173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7161.083706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7162.419490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7163.697300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.585658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7165.509770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7166.764391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.923322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7168.998305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7170.299335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.762702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7172.698996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7174.308947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7175.915626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7177.688509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7179.464703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7181.566424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.866177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7184.679190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7186.497023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7188.856390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7191.057834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.330700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7195.380575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.351979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7199.921665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7202.025482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7204.310428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.819043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7208.364909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7210.206958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7212.179234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7213.841899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7216.351419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7218.854638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7220.781753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7222.599805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7224.191943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7225.601520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7226.949546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.429647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7229.641366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7230.972600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7232.066890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7233.328510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.746449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7235.893383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7236.967486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.233856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7239.916241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7241.473835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7243.245823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.866950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.292157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.629882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7248.932203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7250.178180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7251.454158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7252.794622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7254.185794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7255.789514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7257.051596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7258.325615] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7258.336590] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7258.356539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7259.570203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7261.082173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.462521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7263.599441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7265.135102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7266.393325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7267.478833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.196839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7270.802302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7272.743042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7274.261518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7275.948271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.404618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7278.938581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7280.087717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7281.238330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7282.577965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7284.061519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7285.191519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7286.387196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7287.566358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7289.010664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7290.550976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7292.000029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7293.475413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7295.008217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7296.346158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.663214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7299.713501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7300.941797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7302.795474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7303.927843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7305.027478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.467522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7307.763846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7308.682746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7310.140970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7311.413769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.019077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7314.286891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7315.663664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7317.073263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7318.272266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7319.422947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7320.515278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.015182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.497170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7324.619915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7326.156406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.390435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7328.639563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7330.372910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7331.392155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7332.179729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7333.441349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.746041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7336.093695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.553795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7339.017609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7340.057943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.144286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.538084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7343.875480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7344.920276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7346.154742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7347.232898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7348.415597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7349.918376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7351.135755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.337769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7353.360199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.688103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7355.979896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7357.240139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7358.650161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7360.286174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7361.484462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7362.863135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.508832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7365.537068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.749653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.412537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.293091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7371.486914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7372.961307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7374.350471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7375.428520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7376.355512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7377.633055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.629512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7379.504462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7381.084811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7383.108694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7384.845977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7386.390673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7387.912851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7389.072098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7390.198469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7391.636845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7393.061867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7394.487256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7395.758300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7397.023970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7398.528130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7400.087979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7401.662309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7402.960641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7404.427086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7405.505291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7406.851054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.000349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7409.113155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7410.269686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7411.651405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7413.050456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7414.472477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7415.849081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.129674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7418.325711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7419.566149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.053633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7422.072428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7422.979928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.467947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7425.630286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.970555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.419697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.782353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7431.204938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7432.750192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7434.064558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7435.546411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.975977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7438.490373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7439.869308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.430792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7442.722616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7444.080641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.521217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7446.540933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.910876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7449.405251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7450.558909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7451.807380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7453.182782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7454.316969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7455.343770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7456.527168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7457.864939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7459.215440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.723393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7462.103250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7463.426808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7464.888303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7465.977146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7467.118984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7468.303031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7469.553934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7470.560996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7471.800138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7472.931062] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7472.941808] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7472.967915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7474.207689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7475.389773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.444182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7477.557849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7478.820260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7480.132343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.386548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7482.303419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7483.554631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7484.903405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.228584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7487.346443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7488.507070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7489.840719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7491.095557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.513752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.950359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.417875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7496.784196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7498.155637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7499.539272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7500.745554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7501.988118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7503.481827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7505.016524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.359454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.006988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7509.446394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7510.854060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7512.064317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7513.536133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7514.768236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7516.082946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7517.586806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7519.118553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7520.884239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7522.425523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7524.151824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7525.747042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7527.233512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7528.509626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7530.096221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7531.469206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.618463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7533.633899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7534.706667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7536.075583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.992346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7539.366439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.524060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7541.864431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7543.214520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7544.450965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7545.413367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7546.414782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7547.386404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7548.600177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7549.980078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7551.398019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7553.037319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7554.615821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.393713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7558.096714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7559.968046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7561.423746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7562.689059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7564.159474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7565.638536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7567.170844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7568.776917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7570.284402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7572.055041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7573.234767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7574.445672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7575.661238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.043049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7578.540673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7579.565922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7580.880246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.206492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7583.979382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7585.461793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.603420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7587.612649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7588.862072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7589.733671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7590.745611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7591.708473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7592.916301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7593.752119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7595.382404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7596.960163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7598.116224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7599.746362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7601.120310] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7601.141574] Lustre: Skipped 367 previous similar messages [ 7601.171649] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7601.189834] Lustre: Skipped 367 previous similar messages [ 7601.217036] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7601.232106] Lustre: Skipped 367 previous similar messages [ 7601.245160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7601.284799] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7601.298672] Lustre: Skipped 367 previous similar messages [ 7602.917188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.420320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7605.890919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7606.983619] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7607.002668] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7607.007552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7607.894759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7609.576356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7611.045633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7612.520839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7614.423927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7615.758367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7617.045473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7618.792618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7620.349400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.039332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7623.248298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7624.302430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7625.451802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7626.659269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7627.689565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7628.659385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7629.632100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7630.826151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7632.021765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7633.141804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7634.112960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7635.156292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7636.192336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7637.421391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7638.731572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7639.935161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7641.267847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7642.644987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7644.053174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7645.259798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.410169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7647.827856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7649.006183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7649.905615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7651.116751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7652.307085] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7652.314799] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7652.322810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7653.219208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7654.548120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7656.121186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7657.492097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7658.665691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7660.047859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7661.903425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7663.857823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7665.311479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7666.787288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7667.959306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7669.556777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7671.600228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7673.137525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7674.496367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7675.979318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7677.416591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.678833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7680.090573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7681.242165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7682.473098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7683.724152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7685.016627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7686.456951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7687.862794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7689.348972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7690.705143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7691.892919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7693.377263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7694.865895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7696.214071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7697.390486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7698.796897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7700.120474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7701.534300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7703.106419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7705.003953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7706.552184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7707.780463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7709.074909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7710.423189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7711.695338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7712.652395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7713.762817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7714.977689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.344456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7717.863713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7719.510449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7721.031374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7722.361644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7723.867914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7725.351892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7726.459304] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7726.461894] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7726.474825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7727.889574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7729.188097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7730.141715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7731.461080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7732.933482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7734.366099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7735.708121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7737.154905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7738.644350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7740.050436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7741.563631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7743.119418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7744.815584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7746.379737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7747.878690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7749.230228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7750.888784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7752.027108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7753.175302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7754.383975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7755.777096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7757.017815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7758.323268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7759.646911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7760.979074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7762.422123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7763.654406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7764.670476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7766.112600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7767.448730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7768.264494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7769.423613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7770.654177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7771.946867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7773.180973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7774.114356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7775.289589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7776.431245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7777.691616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7779.043144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7780.261421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7781.545536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7782.638251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7783.787938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7784.959196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7786.190819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7787.830450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7788.927439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7789.942650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7791.544823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7793.178030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7794.429672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7795.879630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7797.481717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7798.490871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7799.860298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7801.163493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7802.317162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7803.680938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7804.858369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7806.191257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7807.383399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7808.795739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7810.138103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7811.672146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7813.158634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7814.100614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7815.369192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7816.365633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7817.536663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.515806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7819.790593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7821.047765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7822.041968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7823.008990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7824.371490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7825.633665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7826.865817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7828.002953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7829.232441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7830.385107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7831.372594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7832.346275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7833.583117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7834.657357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7835.814644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7836.972645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7838.184799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7839.086294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7839.996198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7840.972700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7842.241059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7843.249231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7844.658526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7845.843944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7847.258372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7848.633958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7850.040172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7851.294549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7852.761994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7854.003856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7855.594303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7857.120617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7858.418922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7859.838947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7861.125035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7862.773339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7864.327877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7865.821879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7867.085320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7868.347153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7869.762035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7871.183710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7872.330918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7873.689385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7875.365653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7876.608166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.752798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7879.322574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7880.381872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.681103] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7881.702154] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 7881.719815] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7881.726151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.738207] LustreError: 172414:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 7883.502421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7884.835079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7886.196434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7887.990333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7889.246322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7890.489381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.792935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7893.162877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7894.371360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7895.566922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7897.029716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7898.365203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7899.855186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7901.229416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7902.505546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7903.534593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7904.649584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7906.056154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7907.121409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7908.353102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7909.641960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7910.888116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7912.214903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7913.783732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7915.429483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7917.093847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7918.356666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7920.135619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7921.549891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7923.060770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7924.512922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7925.924465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7926.655146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7927.650296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7928.528174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7929.606774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7931.081429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8475.602324] LustreError: 174800:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 8475.613181] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8475.632745] LustreError: 172414:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 9332.709181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9332.723796] Lustre: Skipped 252 previous similar messages [ 9332.739917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9332.752322] Lustre: Skipped 1 previous similar message [ 9337.832026] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9337.850058] Lustre: Skipped 1 previous similar message [ 9339.631324] Lustre: server umount lustre-OST0000 complete [ 9348.066344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9348.084938] Lustre: Skipped 1 previous similar message [ 9351.848566] Lustre: server umount lustre-MDT0000 complete [ 9353.184854] LustreError: 210553:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9353.205899] LustreError: 210553:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9357.186495] LustreError: 172376:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785795580 with bad export cookie 3073909854866995374 [ 9357.189568] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9357.202163] LustreError: 172376:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9357.882879] Lustre: server umount lustre-MDT0001 complete [ 9373.381330] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 9377.310927] Key type lgssc unregistered [ 9377.726621] LNet: 227786:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9377.742518] LNetError: 227786:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9377.776320] LNet: Removed LNI 192.168.204.141@tcp [ 9379.079249] Key type .llcrypt unregistered [ 9379.084690] Key type ._llcrypt unregistered [ 9411.824257] Key type ._llcrypt registered [ 9411.825820] Key type .llcrypt registered [ 9412.054150] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 9413.780902] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9413.828237] alg: No test for adler32 (adler32-zlib) [ 9414.856878] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 9415.157592] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 9416.903284] Key type lgssc registered [ 9418.611436] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9429.183714] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9429.198335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9430.642571] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9437.294472] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9441.260605] LustreError: 229207:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9441.277269] LustreError: 229207:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9446.376757] LustreError: 229208:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9449.441936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9450.054199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9456.583794] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9466.708823] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9474.627221] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9483.356550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9483.752655] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9492.232746] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9501.188655] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 9501.475791] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9507.333664] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 9509.735522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9509.755940] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 9509.774578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9509.803315] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9509.807900] LustreError: 229227:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 9514.075636] Lustre: lustre-OST0000: Not available for connect from 192.168.204.41@tcp (stopping) [ 9516.533748] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9516.547402] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9516.583546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9516.596659] Lustre: Skipped 1 previous similar message [ 9518.885151] Lustre: server umount lustre-OST0000 complete [ 9529.863192] LustreError: 229227:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 9555.953041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9556.304817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9557.357646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9557.381708] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9557.394406] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9557.405392] Lustre: Skipped 1 previous similar message [ 9564.733503] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9575.737734] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9581.127611] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 9588.509680] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e2647933800.ost_server_uuid 50 [ 9590.186187] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9e2647933800.ost_server_uuid in FULL state after 0 sec [ 9595.361854] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9595.365299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9595.392482] Lustre: Skipped 1 previous similar message [ 9595.395311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9597.831539] Lustre: server umount lustre-OST0000 complete [ 9608.673475] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9608.691314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9610.350962] Lustre: server umount lustre-MDT0000 complete [ 9613.796655] LustreError: 229937:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9613.798050] Lustre: 228518:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785795821/real 1785795821] req@00000000ba69775b x1872542446965376/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1785795837 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 9613.814138] LustreError: 229937:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9613.883475] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9613.903575] Lustre: Skipped 1 previous similar message [ 9615.658300] LustreError: 229188:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785795839 with bad export cookie 1837957624488960769 [ 9615.670523] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9615.673445] LustreError: 229188:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9616.302530] Lustre: server umount lustre-MDT0001 complete [ 9634.307956] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [ 9637.719411] Key type lgssc unregistered [ 9638.127276] LNet: 234418:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9638.143251] LNetError: 234418:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9638.172025] LNet: Removed LNI 192.168.204.141@tcp [ 9639.507389] Key type .llcrypt unregistered [ 9639.510401] Key type ._llcrypt unregistered [ 9683.841890] Key type ._llcrypt registered [ 9683.847156] Key type .llcrypt registered [ 9684.051875] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 9686.303869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9686.327946] alg: No test for adler32 (adler32-zlib) [ 9687.665812] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [ 9688.385235] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [ 9690.311941] Key type lgssc registered [ 9693.662447] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9711.591384] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9711.641550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9713.369546] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9720.746955] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9722.850309] LustreError: 236148:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9722.881646] LustreError: 236148:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9727.989117] LustreError: 236147:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9733.092151] LustreError: 236148:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9733.703389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9734.480301] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9741.632540] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9747.300652] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9758.900652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9759.936299] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9770.079116] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9782.283360] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9782.547911] Lustre: lustre-OST0001: new disk, initializing [ 9782.558539] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9782.574763] Lustre: 238458:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9782.784528] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9783.826761] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 9790.622284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9790.638161] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9790.810164] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9792.592857] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9806.148930] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9813.873978] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9837.027826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9837.055255] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9837.086833] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9837.094513] Lustre: Skipped 2 previous similar messages [ 9837.122474] Lustre: Skipped 3 previous similar messages [ 9841.313893] Lustre: server umount lustre-MDT0000 complete [ 9842.160396] LustreError: 236147:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9842.187713] LustreError: 236147:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9846.908632] LustreError: 236128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785796070 with bad export cookie 1798553539002829669 [ 9846.921815] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9847.276031] LustreError: 238468:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9847.301892] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9847.320312] Lustre: Skipped 1 previous similar message [ 9847.741268] Lustre: server umount lustre-MDT0001 complete [ 9853.584153] Lustre: server umount lustre-OST0000 complete [ 9859.455427] Lustre: server umount lustre-OST0001 complete [ 9891.008378] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [ 9908.001420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9908.966229] LustreError: 241572:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9908.991488] LustreError: 241572:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9909.083024] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9915.773613] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9927.892238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9928.571855] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9935.997790] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9940.073530] Lustre: 242714:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9950.105698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9950.767262] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9952.827209] LustreError: 243071:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9952.866375] LustreError: 243071:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 9953.932617] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 9961.564619] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9974.437676] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9974.772673] Lustre: mylustre-OST0001: Not available for connect from 0@lo (not set up) [ 9986.365468] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [ 9996.670343] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10002.524179] Lustre: 244588:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10020.832545] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10020.841386] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10020.849774] Lustre: Skipped 1 previous similar message [10020.855852] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10020.859716] Lustre: Skipped 1 previous similar message [10021.353565] LustreError: 241567:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10021.370537] LustreError: 241567:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [10021.591780] Lustre: server umount mylustre-MDT0000 complete [10027.082509] LustreError: 241549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785796250 with bad export cookie 1798553539002832441 [10027.084075] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10027.106396] LustreError: 241549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [10027.923218] Lustre: server umount mylustre-MDT0001 complete [10042.825284] Lustre: server umount mylustre-OST0000 complete [10047.967133] Lustre: 235410:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785796255/real 1785796255] req@ffff960068db7100 x1872542733346560/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785796271 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10048.035672] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10048.065205] Lustre: Skipped 3 previous similar messages [10048.471188] Lustre: server umount mylustre-OST0001 complete [10074.683967] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [10087.107471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10087.869312] LustreError: 246897:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10087.903481] LustreError: 246897:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [10088.011879] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10088.014144] Lustre: Skipped 1 previous similar message [10094.759436] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10107.109944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10113.294435] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10117.118569] Lustre: 248041:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10125.513474] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10133.422557] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [10135.826284] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10146.291894] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10155.227766] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10164.353343] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10168.693966] Lustre: 249910:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10182.642488] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10182.652584] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [10182.674658] Lustre: Skipped 3 previous similar messages [10182.693743] Lustre: Skipped 3 previous similar messages [10186.952830] Lustre: server umount tfs-MDT0000 complete [10191.903318] LustreError: 246873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785796415 with bad export cookie 1798553539002834107 [10191.908723] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10191.917152] LustreError: 246873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10192.551530] Lustre: server umount tfs-MDT0001 complete [10206.916599] Lustre: server umount tfs-OST0000 complete [10208.737290] Lustre: 235411:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785796416/real 1785796416] req@ffff96006491f100 x1872542733403904/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785796432 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10208.782528] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10212.757205] Lustre: server umount tfs-OST0001 complete [10243.406769] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [10259.076216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10259.918446] LustreError: 252167:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10259.967780] LustreError: 252167:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 13 previous similar messages [10260.080914] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10260.100402] Lustre: Skipped 3 previous similar messages [10268.484748] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10282.380684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10288.179538] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10294.189249] Lustre: 253309:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10308.802223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10317.845223] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10328.960487] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10330.582374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [10338.587605] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10349.515110] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10354.769709] Lustre: 255183:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10363.209966] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 18:36:24 (1785796584) [10381.828466] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10381.855420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10381.863924] Lustre: Skipped 2 previous similar messages [10381.886389] Lustre: Skipped 3 previous similar messages [10388.644458] Lustre: server umount lustre-MDT0000 complete [10394.243371] LustreError: 252142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785796617 with bad export cookie 1798553539002835549 [10394.257386] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10394.274607] LustreError: 252142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10397.155081] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10397.175664] Lustre: Skipped 5 previous similar messages [10401.393529] Lustre: server umount lustre-MDT0001 complete [10413.728964] Lustre: server umount lustre-OST0000 complete [10426.854640] Lustre: server umount lustre-OST0001 complete [10437.223875] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [10449.704190] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [10504.207317] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [10504.207317] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10504.207317] [10504.270430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10504.730232] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10504.789084] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10504.980373] Lustre: lustre-MDT0000: new disk, initializing [10505.152687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10505.161445] Lustre: Skipped 3 previous similar messages [10505.202960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10511.922381] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10527.548603] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [10527.548603] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10527.548603] [10527.601169] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10527.852368] Lustre: 258373:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10528.008787] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10528.014990] Lustre: Skipped 1 previous similar message [10528.179433] Lustre: lustre-MDT0001: new disk, initializing [10528.395829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10528.406131] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10535.085994] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10549.534075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10550.161481] Lustre: lustre-OST0000: new disk, initializing [10550.168153] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10550.184170] Lustre: 259959:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10551.742388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10551.765800] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10552.043712] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10560.019288] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10579.553323] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10579.838428] Lustre: lustre-OST0001: new disk, initializing [10579.860421] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10579.867702] Lustre: 260832:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10581.549972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10581.571118] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10581.901985] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10594.709973] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10615.115735] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 18:40:35 (1785796835) [10638.290898] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 18:40:59 (1785796859) [10658.791650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10658.803955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10658.810727] Lustre: Skipped 4 previous similar messages [10658.831030] Lustre: Skipped 3 previous similar messages [10663.549717] Lustre: server umount lustre-OST0000 complete [10663.920923] LustreError: 259945:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10663.962989] LustreError: 259945:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [10671.586561] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10677.477874] Lustre: server umount lustre-MDT0000 complete [10682.580652] LustreError: 258366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785796906 with bad export cookie 1798553539002836879 [10682.586138] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10682.590588] LustreError: 258366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10689.179936] Lustre: server umount lustre-MDT0001 complete [10705.417814] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [10720.787392] Lustre: server umount lustre-OST0001 complete [10731.156724] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [10754.328640] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [10807.752211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10808.201415] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10808.264535] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10808.420458] Lustre: lustre-MDT0000: new disk, initializing [10808.563071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10808.574632] Lustre: Skipped 3 previous similar messages [10808.625756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10816.281520] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10831.836651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10832.102952] Lustre: 265420:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10832.189932] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10832.203485] Lustre: Skipped 1 previous similar message [10832.428980] Lustre: lustre-MDT0001: new disk, initializing [10832.793945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10832.816708] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10839.879528] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10854.121153] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10861.622702] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10870.228582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10870.648392] Lustre: lustre-OST0000: new disk, initializing [10870.655566] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10870.675456] Lustre: 267390:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10871.749993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10871.761920] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10871.951922] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10878.664076] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [10890.988610] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10902.498996] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10902.503391] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10902.514491] LustreError: Skipped 1 previous similar message [10902.518055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10902.518061] Lustre: Skipped 6 previous similar messages [10902.565138] Lustre: Skipped 5 previous similar messages [10906.193673] Lustre: server umount lustre-OST0000 complete [10919.139179] Lustre: server umount lustre-MDT0000 complete [10924.338501] LustreError: 265413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785797147 with bad export cookie 1798553539002838349 [10924.342941] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10924.355652] LustreError: 265413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10925.001724] Lustre: server umount lustre-MDT0001 complete [10941.105365] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [10944.747525] Key type lgssc unregistered [10945.385492] LNet: 269152:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10945.421774] LNetError: 269152:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10945.445720] LNet: Removed LNI 192.168.204.141@tcp [10946.812120] Key type .llcrypt unregistered [10946.814535] Key type ._llcrypt unregistered [10983.044178] Key type ._llcrypt registered [10983.049329] Key type .llcrypt registered [10983.248227] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [10985.092567] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10985.137899] alg: No test for adler32 (adler32-zlib) [10986.291992] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [10986.568912] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [10988.279179] Key type lgssc registered [10989.720400] Lustre: Echo OBD driver; http://www.lustre.org/ [10999.426431] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10999.483955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11001.335745] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11007.490876] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11011.561348] LustreError: 270625:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11011.593684] LustreError: 270625:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11016.684397] LustreError: 270626:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11019.034295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11019.558448] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11026.567587] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11036.356744] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11042.841688] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11051.600888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11051.900922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11060.298420] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11071.642826] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11072.443934] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [11073.704579] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [11077.602601] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11077.606574] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11077.625394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11077.670249] Lustre: Skipped 1 previous similar message [11080.207476] Lustre: server umount lustre-OST0000 complete [11091.424526] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11091.443965] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11091.480462] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11091.488289] Lustre: Skipped 1 previous similar message [11092.963255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11092.968067] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11095.514346] Lustre: server umount lustre-MDT0000 complete [11098.081986] LustreError: 270626:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11098.095883] LustreError: 270626:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11100.012732] LustreError: 270605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785797323 with bad export cookie 4914647112472546884 [11100.016431] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11100.026771] LustreError: 270605:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11100.853500] Lustre: server umount lustre-MDT0001 complete [11115.101192] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [11119.660629] Key type lgssc unregistered [11120.255323] LNet: 274213:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11120.267204] LNetError: 274213:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11120.302528] LNet: Removed LNI 192.168.204.141@tcp [11121.759292] Key type .llcrypt unregistered [11121.760633] Key type ._llcrypt unregistered [11149.008180] Key type ._llcrypt registered [11149.010206] Key type .llcrypt registered [11149.131838] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [11151.206306] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11151.294456] alg: No test for adler32 (adler32-zlib) [11152.610412] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [11153.085231] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [11154.928122] Key type lgssc registered [11156.820639] Lustre: Echo OBD driver; http://www.lustre.org/ [11168.966846] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11169.027360] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11184.799369] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [11189.919387] LustreError: 275428:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.141@tcp: failed processing log, type 4: rc = -110 [11216.671233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11226.263956] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11240.096795] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11243.513969] Lustre: server umount lustre-OST0000 complete [11273.560733] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [11276.856894] Key type lgssc unregistered [11277.252659] LNet: 277085:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11277.263711] LNetError: 277085:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11277.304415] LNet: Removed LNI 192.168.204.141@tcp [11278.405371] Key type .llcrypt unregistered [11278.406710] Key type ._llcrypt unregistered [11338.374873] Key type ._llcrypt registered [11338.378890] Key type .llcrypt registered [11338.508334] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [11340.097838] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11340.175453] alg: No test for adler32 (adler32-zlib) [11341.324234] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [11341.789573] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [11343.663455] Key type lgssc registered [11345.268603] Lustre: Echo OBD driver; http://www.lustre.org/ [11362.448930] loop0: detected capacity change from 0 to 204800000 [11362.875199] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11362.896511] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11363.681244] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11365.349480] loop0: detected capacity change from 0 to 204800000 [11365.705699] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11365.728543] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11366.750841] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11368.527215] loop0: detected capacity change from 0 to 204800000 [11368.903176] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11368.925588] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11369.256195] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11371.352084] loop0: detected capacity change from 0 to 204800000 [11371.768365] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11371.793769] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11372.181690] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11374.664736] loop0: detected capacity change from 0 to 204800000 [11374.735762] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11381.769286] loop0: detected capacity change from 0 to 204800000 [11381.844149] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11387.899226] loop0: detected capacity change from 0 to 204800000 [11387.940030] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11394.127094] loop0: detected capacity change from 0 to 204800000 [11394.214463] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11399.893441] loop0: detected capacity change from 0 to 204800000 [11399.968720] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11400.049548] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11400.125860] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11400.132315] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [11400.138852] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [11400.149411] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [11400.158149] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [11400.167259] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [11400.177747] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [11400.186680] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [11400.193510] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [11400.201543] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [11400.209414] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [11400.220440] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [11400.230683] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [11400.238442] Lustre: 280510:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [11407.329933] Lustre: server umount MGS complete [11408.487234] loop0: detected capacity change from 0 to 204800000 [11408.630762] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11410.112206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11410.204670] Lustre: lustre-MDT0000: Aborting client recovery [11411.494463] loop1: detected capacity change from 0 to 204800000 [11411.758368] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11411.896617] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11411.912199] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [11411.921530] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [11411.931465] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [11411.939168] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [11411.946625] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [11411.958632] Lustre: 280941:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [11411.974185] Lustre: Found index 1 for lustre-MDT0001, updating log [11412.132052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11412.178750] Lustre: lustre-MDT0001: Aborting client recovery [11413.323044] loop2: detected capacity change from 0 to 204800000 [11413.515190] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11413.606293] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11413.629541] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11413.645863] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11413.670479] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [11413.690497] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [11413.711911] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11413.742513] Lustre: 281021:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [11414.013880] Lustre: Found index 0 for lustre-OST0000, updating log [11414.117059] Lustre: 281039:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11414.233296] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11414.251328] Lustre: lustre-OST0000: Aborting recovery [11415.506285] loop3: detected capacity change from 0 to 204800000 [11415.680563] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11415.809396] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11415.828509] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [11415.851797] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11415.882559] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11415.908742] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [11415.925615] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [11415.942677] Lustre: 281105:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11415.992843] Lustre: Found index 1 for lustre-OST0001, updating log [11416.090546] Lustre: 281111:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11416.225031] Lustre: lustre-OST0001: Aborting recovery [11416.256604] LustreError: 280885:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [11420.326077] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11420.352325] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11420.433348] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [11420.440158] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [11422.376218] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [11422.381170] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [11425.589048] LustreError: 280852:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 583us: rc = -2 [11426.103841] LustreError: 280852:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2212us: rc = -2 [11426.125530] LustreError: 280852:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [11443.177109] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11443.190914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11443.197120] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11443.210972] Lustre: Skipped 3 previous similar messages [11443.220986] Lustre: Skipped 2 previous similar messages [11446.350600] Lustre: server umount lustre-MDT0000 complete [11447.530635] LustreError: 280835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785797670 with bad export cookie 18265835031525004123 [11447.544378] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11447.550956] LustreError: 280835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11448.287659] LustreError: 280850:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11448.289431] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11448.295271] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11448.295280] Lustre: Skipped 1 previous similar message [11448.319885] LustreError: 280850:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11448.368108] Lustre: Skipped 1 previous similar message [11452.321098] LustreError: 281119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11452.322035] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11452.340637] LustreError: 281119:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11452.361919] Lustre: Skipped 2 previous similar messages [11454.375202] Lustre: server umount lustre-MDT0001 complete [11461.864430] Lustre: server umount lustre-OST0000 complete [11469.747424] Lustre: server umount lustre-OST0001 complete [11480.896980] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 18:55:01 (1785797701) [11530.722817] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [11554.649441] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [11606.047106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11606.569664] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11606.622365] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11606.797503] Lustre: lustre-MDT0000: new disk, initializing [11606.995352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11606.997657] Lustre: Skipped 1 previous similar message [11607.035024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11614.493226] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11633.083944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11633.421779] Lustre: 285208:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11633.635908] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11633.652814] Lustre: Skipped 1 previous similar message [11633.988897] Lustre: lustre-MDT0001: new disk, initializing [11634.270621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11634.361589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11634.387779] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11641.205617] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11653.471941] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11661.697590] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11671.145472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11671.644885] Lustre: lustre-OST0000: new disk, initializing [11671.651714] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11671.670292] Lustre: 287177:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11671.827892] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11680.867210] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11680.886182] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11681.038146] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11681.212487] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11694.365437] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11703.372314] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [11713.001753] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [11719.084927] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [11768.292713] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11768.293118] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11768.315242] LustreError: Skipped 1 previous similar message [11768.317279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11768.349464] Lustre: Skipped 1 previous similar message [11768.382405] Lustre: Skipped 1 previous similar message [11771.481511] Lustre: server umount lustre-OST0000 complete [11777.504410] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11777.517811] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11777.552220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11783.187367] Lustre: server umount lustre-MDT0000 complete [11783.653085] LustreError: 285214:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11783.684287] LustreError: 285214:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11787.682345] LustreError: 285199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785798011 with bad export cookie 18265835031525006832 [11787.683455] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11787.697955] LustreError: 285199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11788.339221] Lustre: server umount lustre-MDT0001 complete [11799.738175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11805.202682] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11810.993437] Lustre: server umount MGS complete [11825.139945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11825.994744] LustreError: 290908:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11826.017641] LustreError: 290921:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [11826.157084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11831.264271] LustreError: 290909:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11831.283590] LustreError: 290909:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11832.112881] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11841.517807] LustreError: 290908:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11841.536104] LustreError: 290908:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11845.991420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11846.627220] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11846.638554] Lustre: Skipped 2 previous similar messages [11855.151801] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11865.714332] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11874.915724] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11884.130977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11884.640935] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11884.652031] Lustre: Skipped 1 previous similar message [11893.301829] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [11903.523381] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11939.311446] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11939.326431] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11939.359454] Lustre: Skipped 1 previous similar message [11939.364886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11944.947691] Lustre: server umount lustre-OST0000 complete [11952.610000] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11952.629309] Lustre: Skipped 1 previous similar message [11957.468351] Lustre: server umount lustre-MDT0000 complete [11957.730207] LustreError: 290908:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11957.743496] LustreError: 290908:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11964.020115] LustreError: 290889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785798187 with bad export cookie 18265835031525008764 [11964.026369] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11964.040734] LustreError: 290889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11965.172767] Lustre: server umount lustre-MDT0001 complete [11991.832285] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [11998.019419] Key type lgssc unregistered [11998.693077] LNet: 294898:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11998.697730] LNetError: 294898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11998.726640] LNet: Removed LNI 192.168.204.141@tcp [12000.142469] Key type .llcrypt unregistered [12000.150576] Key type ._llcrypt unregistered [12049.256367] Key type ._llcrypt registered [12049.261505] Key type .llcrypt registered [12049.373856] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [12073.584690] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [12077.098533] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12078.019218] alg: No test for adler32 (adler32-zlib) [12079.843162] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [12080.523597] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [12082.716703] Key type lgssc registered [12084.356387] Lustre: Echo OBD driver; http://www.lustre.org/ [12142.129980] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12142.173687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12143.579210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12143.622205] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12143.737271] Lustre: lustre-MDT0000: new disk, initializing [12143.924456] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12143.947226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12151.144948] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12166.204266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12166.444488] Lustre: 298863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12166.554647] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12166.568706] Lustre: Skipped 1 previous similar message [12166.734133] Lustre: lustre-MDT0001: new disk, initializing [12166.955983] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12167.050379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12167.068417] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12175.864491] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12188.290259] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12195.408391] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12205.291859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12205.570513] Lustre: lustre-OST0000: new disk, initializing [12205.588720] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12205.602477] Lustre: 300835:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12205.808660] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12211.810876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12211.820947] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12211.941403] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12218.153620] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12231.296198] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12240.488133] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12254.892551] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12264.720727] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12314.596628] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12314.600151] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12314.622313] LustreError: Skipped 1 previous similar message [12314.651471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12317.875671] Lustre: server umount lustre-OST0000 complete [12324.833444] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12324.849502] Lustre: Skipped 1 previous similar message [12324.856599] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12324.870049] Lustre: Skipped 2 previous similar messages [12329.965407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12329.975944] Lustre: Skipped 1 previous similar message [12335.074306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12335.080494] Lustre: Skipped 1 previous similar message [12337.119136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12337.531096] Lustre: server umount lustre-MDT0000 complete [12343.536019] LustreError: 301692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785798566 with bad export cookie 6369026657406035228 [12343.549277] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12343.553340] LustreError: 301692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12344.351410] Lustre: server umount lustre-MDT0001 complete [12356.887658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12364.190479] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12370.850971] Lustre: server umount MGS complete [12386.384361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12387.104728] LustreError: 304756:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12387.124658] LustreError: 304756:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12387.143371] LustreError: 304770:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [12387.249814] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12392.416776] LustreError: 304757:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12392.625855] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12397.541197] LustreError: 304756:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12402.668857] LustreError: 304757:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12407.782294] LustreError: 304756:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12408.074267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12409.263559] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12416.462953] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12426.492726] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12433.118792] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12444.874697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12445.567904] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12457.131323] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12471.491618] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12508.135948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12508.138328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12508.151984] Lustre: Skipped 2 previous similar messages [12508.175019] Lustre: Skipped 1 previous similar message [12513.066465] Lustre: server umount lustre-OST0000 complete [12518.371927] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12518.388407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12518.396446] Lustre: Skipped 1 previous similar message [12532.191690] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12532.639367] Lustre: server umount lustre-MDT0000 complete [12532.711190] LustreError: 304756:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12537.800837] LustreError: 304737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785798761 with bad export cookie 6369026657406037069 [12537.822859] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12537.823905] LustreError: 304737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12538.333275] Lustre: server umount lustre-MDT0001 complete [12553.906308] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [12557.657812] Key type lgssc unregistered [12558.312022] LNet: 308709:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12558.345295] LNetError: 308709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12559.420603] LNet: Removed LNI 192.168.204.141@tcp [12562.038564] Key type .llcrypt unregistered [12562.044698] Key type ._llcrypt unregistered [12602.369903] Key type ._llcrypt registered [12602.373182] Key type .llcrypt registered [12602.557317] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [12620.518163] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [12622.298945] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12622.537410] alg: No test for adler32 (adler32-zlib) [12623.968389] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [12624.546407] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [12626.359189] Key type lgssc registered [12628.132262] Lustre: Echo OBD driver; http://www.lustre.org/ [12664.993088] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [12685.048857] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12685.099327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12686.720194] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12686.795445] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12687.073941] Lustre: lustre-MDT0000: new disk, initializing [12687.234131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12687.297512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12694.108427] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [12707.129934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12707.492277] Lustre: lustre-OST0000: new disk, initializing [12707.495692] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12707.500072] Lustre: Skipped 1 previous similar message [12707.505357] Lustre: 312779:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12707.567729] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12715.731848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [12715.760567] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [12715.800833] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [12717.487404] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [13826.552788] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13827.234065] Lustre: server umount lustre-MDT0000 complete [13832.274209] LustreError: 311965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785800055 with bad export cookie 17264801099409851111 [13832.300209] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13832.798558] Lustre: server umount lustre-OST0000 complete [13876.932128] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [13892.595835] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [13941.362085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13941.735183] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13941.773669] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13941.895508] Lustre: lustre-MDT0000: new disk, initializing [13942.013688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13942.041142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13948.061946] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [13962.924624] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13963.098201] Lustre: 317170:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13963.149969] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13963.153524] Lustre: Skipped 1 previous similar message [13963.328906] Lustre: lustre-MDT0001: new disk, initializing [13963.413258] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13963.449412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13963.470353] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13970.367785] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [13987.498206] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13999.296363] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14011.617130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14012.139185] Lustre: lustre-OST0000: new disk, initializing [14012.150720] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14012.166131] Lustre: 319143:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14012.321243] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14020.225802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14020.242997] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14020.449745] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14023.313940] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14035.774989] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14041.965859] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14042.247072] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14048.285784] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [14048.673530] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [14056.416258] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14056.421974] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14056.430412] LustreError: Skipped 1 previous similar message [14056.436563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14056.451768] Lustre: Skipped 1 previous similar message [14056.476295] Lustre: Skipped 1 previous similar message [14057.793768] Lustre: server umount lustre-OST0000 complete [14066.151395] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14066.156372] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14066.169316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14069.236098] Lustre: server umount lustre-MDT0000 complete [14071.781771] LustreError: 318237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14071.805560] LustreError: 318237:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [14074.413785] LustreError: 317163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785800297 with bad export cookie 17264801099412791930 [14074.425225] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14074.431042] LustreError: 317163:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [14075.115103] Lustre: server umount lustre-MDT0001 complete [14080.222660] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [14082.474630] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:38:23 (1785800303) [14091.581128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14091.936366] LustreError: 321457:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14092.011824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14096.702854] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14097.379218] LustreError: 321458:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14102.503035] LustreError: 321457:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14107.624826] LustreError: 321458:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14110.187419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14110.512819] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14117.372581] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14127.978631] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14135.824627] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14146.750649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14147.134953] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14157.129879] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14168.583685] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14177.534599] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14177.800362] Lustre: lustre-OST0001: new disk, initializing [14177.811330] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [14177.820505] Lustre: 324299:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14177.975577] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [14186.734915] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14187.137501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [14187.144800] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [14187.197649] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14188.259249] LustreError: 324683:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14188.272137] LustreError: 324683:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [14189.284034] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14191.335179] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14191.347602] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [14195.502214] LustreError: 324683:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14195.528889] LustreError: 324683:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [14202.946337] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14203.483238] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14203.707914] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14203.725888] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [14213.558468] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e2678b5d000.ost_server_uuid 50 [14215.754927] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9e2678b5d000.ost_server_uuid in FULL state after 0 sec [14220.470906] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14220.481428] LustreError: 324679:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 30 previous similar messages [14223.554588] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9e2678b5d000.ost_server_uuid 50 [14225.284805] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9e2678b5d000.ost_server_uuid in IDLE state after 0 sec [14227.729528] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [14243.305253] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [14243.313780] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14243.317629] LustreError: Skipped 1 previous similar message [14243.319260] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [14243.319264] Lustre: Skipped 1 previous similar message [14243.389477] Lustre: Skipped 2 previous similar messages [14248.705590] Lustre: server umount lustre-OST0001 complete [14248.929995] LustreError: 323283:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14248.956062] LustreError: 323283:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14259.179121] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14259.196326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14259.196679] Lustre: Skipped 1 previous similar message [14259.215041] Lustre: Skipped 3 previous similar messages [14264.304919] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14264.312467] Lustre: Skipped 1 previous similar message [14265.687401] Lustre: server umount lustre-OST0000 complete [14274.530288] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14274.531304] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14274.541394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14274.571317] Lustre: Skipped 1 previous similar message [14278.068427] Lustre: server umount lustre-MDT0000 complete [14279.652301] LustreError: 325104:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14279.693955] LustreError: 325104:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [14284.091441] LustreError: 321437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785800507 with bad export cookie 17264801099412792812 [14284.093787] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14284.119901] LustreError: 321437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14285.497185] Lustre: server umount lustre-MDT0001 complete [14305.006432] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [14309.269083] Key type lgssc unregistered [14309.705053] LNet: 327004:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14309.728434] LNetError: 327004:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14309.749830] LNet: Removed LNI 192.168.204.141@tcp [14310.612734] Key type .llcrypt unregistered [14310.616937] Key type ._llcrypt unregistered [14343.500232] Key type ._llcrypt registered [14343.505935] Key type .llcrypt registered [14343.722944] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [14345.558551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [14345.652180] alg: No test for adler32 (adler32-zlib) [14347.233432] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [14347.891245] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [14349.543163] Key type lgssc registered [14352.375196] Lustre: Echo OBD driver; http://www.lustre.org/ [14363.853806] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [14363.871908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14365.395633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14370.720983] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14374.952916] LustreError: 328425:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14374.981779] LustreError: 328425:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [14380.001958] LustreError: 328424:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14381.612567] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14385.125855] LustreError: 328425:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14390.247027] LustreError: 328424:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14392.602920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14393.548535] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14400.582612] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14412.992215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14413.549041] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14417.693484] LustreError: 330107:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14422.687217] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14428.147122] LustreError: 330106:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14428.173842] LustreError: 330106:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14432.899549] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14435.336156] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [14442.416754] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14442.886804] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [14448.154188] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [14452.287636] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14465.726105] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14474.949854] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [14475.321593] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [14591.464204] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14591.503695] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [14591.514786] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14591.523040] Lustre: Skipped 1 previous similar message [14594.853552] Lustre: server umount lustre-OST0001 complete [14596.584362] LustreError: 330106:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14596.613273] LustreError: 330106:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [14606.824410] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14606.850543] Lustre: Skipped 1 previous similar message [14606.863749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14606.871754] Lustre: Skipped 1 previous similar message [14612.080507] Lustre: server umount lustre-OST0000 complete [14618.597945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14618.612815] LustreError: Skipped 1 previous similar message [14618.623143] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14618.652474] Lustre: Skipped 1 previous similar message [14618.674679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14622.176262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14622.177706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14622.214399] Lustre: Skipped 1 previous similar message [14623.868851] Lustre: server umount lustre-MDT0000 complete [14628.494617] LustreError: 328404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785800851 with bad export cookie 10237008264426820749 [14628.525466] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14629.284860] Lustre: server umount lustre-MDT0001 complete [14644.533959] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [14650.100947] Key type lgssc unregistered [14651.016700] LNet: 333378:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14651.040476] LNetError: 333378:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14652.070877] LNet: Removed LNI 192.168.204.141@tcp [14654.301102] Key type .llcrypt unregistered [14654.304819] Key type ._llcrypt unregistered [14717.440380] Key type ._llcrypt registered [14717.444629] Key type .llcrypt registered [14717.599707] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [14718.948361] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [14718.966245] alg: No test for adler32 (adler32-zlib) [14720.114523] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [14720.416931] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [14722.143149] Key type lgssc registered [14724.051561] Lustre: Echo OBD driver; http://www.lustre.org/ [14736.201909] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [14736.230452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14737.907611] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14743.787345] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14748.133324] LustreError: 335671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14748.154062] LustreError: 335671:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [14753.252316] LustreError: 335672:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14754.153266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14754.613559] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14760.905086] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14763.994184] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [14771.274235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14771.668734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14778.869883] LustreError: 337160:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14780.762587] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14783.094268] LustreError: 337468:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14783.130984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [14783.132282] LustreError: 337468:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14788.581803] LustreError: 337604:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14788.618841] LustreError: 337604:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [14792.742101] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14793.292873] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [14798.358651] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [14802.063892] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14815.059297] Lustre: DEBUG MARKER: Using TIMEOUT=20 [14827.859661] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [14852.066677] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14852.081172] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14852.100273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14854.481961] Lustre: server umount lustre-MDT0000 complete [14854.633170] LustreError: 335671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14854.635489] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14854.717417] Lustre: Skipped 2 previous similar messages [14860.367041] LustreError: 335652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785801083 with bad export cookie 16884043709870730780 [14860.387273] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14860.395111] LustreError: 335652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14861.055171] Lustre: server umount lustre-MDT0001 complete [14877.839793] Lustre: server umount lustre-OST0000 complete [14880.736927] Lustre: 334934:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785801088/real 1785801088] req@ffff960053df2300 x1872548009685248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785801104 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [14880.805049] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14882.335162] Lustre: 334935:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785801089/real 1785801089] req@ffff960053df1500 x1872548009685504/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785801105 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [14885.983228] Lustre: 334934:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785801093/real 1785801093] req@ffff9601434df480 x1872548009686016/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785801109 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [14887.345871] Lustre: server umount lustre-OST0001 complete [14916.857702] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [14936.306221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14937.481433] LustreError: 340945:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14937.522903] LustreError: 340945:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [14937.766342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14945.488718] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14958.475623] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14959.020658] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14964.868819] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [14968.184877] Lustre: 342087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [14978.607471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14979.251458] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14983.348094] LustreError: 342442:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14983.380341] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [14983.394024] LustreError: 342442:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [14989.665242] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15002.864966] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15008.785567] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [15017.630739] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15029.940238] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15034.217912] Lustre: 343963:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [15044.580772] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15044.587098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15044.609541] Lustre: Skipped 3 previous similar messages [15044.633935] Lustre: Skipped 3 previous similar messages [15048.324815] Lustre: server umount lustre-MDT0000 complete [15049.699378] LustreError: 340945:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15049.746687] LustreError: 340945:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [15054.571958] LustreError: 343965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785801278 with bad export cookie 16884043709870732481 [15054.586118] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15054.618812] LustreError: 343965:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [15054.820373] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15054.869865] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [15054.880632] Lustre: Skipped 1 previous similar message [15055.152626] Lustre: server umount lustre-MDT0001 complete [15063.056172] Lustre: server umount lustre-OST0000 complete [15071.916658] Lustre: server umount lustre-OST0001 complete [15080.501425] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [15083.433667] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [15085.877709] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 19:55:06 (1785801306) [15121.807701] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [15132.834516] loop0: detected capacity change from 0 to 1099511627776 [15133.557802] SGI XFS with ACLs, security attributes, quota, no debug enabled [15133.562583] *** VALIDATE xfs *** [15133.595143] XFS (loop0): Mounting V5 Filesystem [15133.663383] XFS (loop0): Ending clean mount [15135.590514] loop1: detected capacity change from 0 to 18691697672192 [15304.971447] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [15318.811835] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [15375.810923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15376.236410] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [15376.397502] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [15376.677779] Lustre: lustre-MDT0000: new disk, initializing [15376.838446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15376.840874] Lustre: Skipped 1 previous similar message [15376.883864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [15384.537688] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15397.600206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15397.862415] Lustre: 349136:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [15398.015677] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [15398.027115] Lustre: Skipped 1 previous similar message [15398.169785] Lustre: lustre-MDT0001: new disk, initializing [15398.383553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [15398.396479] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [15405.493228] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15418.703336] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15426.347390] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [15436.240544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15436.903899] Lustre: lustre-OST0000: new disk, initializing [15436.924435] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [15436.934446] Lustre: 351104:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [15439.023880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [15439.040569] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [15439.270484] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [15447.577342] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15461.948857] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15469.890136] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [15470.389404] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [15475.707075] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [15476.099703] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [15480.310248] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [15480.324041] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15480.325544] LustreError: Skipped 1 previous similar message [15480.333832] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15480.392195] Lustre: Skipped 2 previous similar messages [15485.493765] LustreError: 351088:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15485.525207] LustreError: 351088:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [15485.872507] Lustre: server umount lustre-OST0000 complete [15495.649117] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15495.652118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15495.673123] Lustre: Skipped 2 previous similar messages [15505.375294] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [15505.663539] Lustre: server umount lustre-MDT0000 complete [15509.493387] LustreError: 349126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785801732 with bad export cookie 16884043709870734035 [15509.509119] LustreError: 349126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [15509.513222] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15510.310899] Lustre: server umount lustre-MDT0001 complete [15521.023842] XFS (loop0): Unmounting Filesystem [15526.766584] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:02:27 (1785801747) [15537.393669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15538.418556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15538.429927] Lustre: Skipped 2 previous similar messages [15544.585763] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15558.935602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15568.790212] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15578.406356] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15586.867160] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [15595.839351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15606.084847] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15618.379608] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15633.377826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15633.390693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15633.395293] Lustre: Skipped 3 previous similar messages [15633.406619] Lustre: Skipped 3 previous similar messages [15638.366678] Lustre: server umount lustre-OST0000 complete [15646.689546] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15652.235983] Lustre: server umount lustre-MDT0000 complete [15658.236525] LustreError: 353661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785801881 with bad export cookie 16884043709870734917 [15658.259229] LustreError: 353661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [15658.285147] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15659.214851] Lustre: server umount lustre-MDT0001 complete [15675.000866] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [15679.172879] Key type lgssc unregistered [15679.761496] LNet: 357312:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15679.772519] LNetError: 357312:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [15679.813727] LNet: Removed LNI 192.168.204.141@tcp [15681.736163] Key type .llcrypt unregistered [15681.737810] Key type ._llcrypt unregistered [15721.474069] Key type ._llcrypt registered [15721.476710] Key type .llcrypt registered [15721.628529] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [15722.637610] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [15722.716926] alg: No test for adler32 (adler32-zlib) [15723.677387] Lustre: Lustre: Build Version: 2.17.56_1_g3060c05 [15724.013344] LNet: Added LNI 192.168.204.141@tcp [8/256/0/180] [15725.687204] Key type lgssc registered [15727.058408] Lustre: Echo OBD driver; http://www.lustre.org/ [15744.472460] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [15744.530191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15746.140809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15751.932304] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15756.267615] LustreError: 358893:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15756.290818] LustreError: 358893:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [15761.380194] LustreError: 358892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15762.825721] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15763.276476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15768.978666] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15779.662601] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15788.344994] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [15800.643076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15801.535758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15813.320026] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15824.760373] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15835.105740] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [15835.122996] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15835.147139] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [15836.640909] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [15836.660864] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15838.296154] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.41@tcp (stopping) [15838.298443] Lustre: Skipped 1 previous similar message [15841.157130] Lustre: server umount lustre-MDT0001 complete [15841.773636] LustreError: 358893:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15846.882640] LustreError: 358886:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15846.937139] LustreError: 358886:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [15852.015378] LustreError: 358893:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15852.052765] LustreError: 358893:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [15855.865672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15856.069965] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [15856.085330] Lustre: Found index 1 for lustre-MDT0001, updating log [15856.248753] Lustre: 358882:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [15856.907489] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15862.265028] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [15862.848590] LustreError: 362287:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [15862.861917] LustreError: 362287:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [15862.895371] LustreError: 358884:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [15863.054701] LustreError: 362292:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.141@tcp: cfg command failed: rc = -17 [15863.077774] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [15863.077774] [15864.284854] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15877.638430] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [15877.640268] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15877.687141] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [15882.720914] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [15882.735785] Lustre: Skipped 2 previous similar messages [15884.343351] LustreError: 359848:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [15884.370149] LustreError: 359848:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [15884.585270] Lustre: server umount lustre-MDT0001 complete [15900.297526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15900.391105] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [15900.404135] Lustre: Found index 1 for lustre-MDT0001, updating log [15900.474403] Lustre: 358882:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [15900.816752] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15906.292081] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [15907.227934] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [15907.873642] LustreError: 363443:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [15907.894695] LustreError: 363443:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [15907.908908] LustreError: 358884:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [15907.926608] LustreError: 358884:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [15908.038088] LustreError: 363451:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [15908.051912] LustreError: 363451:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [15908.063808] LustreError: 363451:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.141@tcp: cfg command failed: rc = -17 [15908.079868] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.141@tcp [15908.079868] [16217.168056] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.41@tcp (stopping) [16217.176265] Lustre: Skipped 1 previous similar message [16218.595573] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16218.600975] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16218.607281] Lustre: Skipped 1 previous similar message [16221.333377] Lustre: server umount lustre-MDT0001 complete [16222.259830] LustreError: 361526:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16222.274464] LustreError: 361526:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [16235.771744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16235.919465] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16235.930911] Lustre: Found index 1 for lustre-MDT0001, updating log [16235.979573] Lustre: 358880:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16236.379897] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [16241.649117] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16241.924680] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [16242.180934] LustreError: 370609:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16242.191406] LustreError: 370609:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16242.205525] LustreError: 358884:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16242.212661] LustreError: 358884:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [16242.285687] LustreError: 370615:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [16242.302478] LustreError: 370615:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.141@tcp: cfg command failed: rc = -114 [16242.316436] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [16242.316436] [16569.333164] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16569.355324] Lustre: Skipped 1 previous similar message [16569.356030] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [16569.365719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16569.379529] Lustre: Skipped 4 previous similar messages [16574.439051] LustreError: 359848:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16574.474124] LustreError: 359848:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [16574.703111] Lustre: server umount lustre-MDT0000 complete [16580.414488] LustreError: 358873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785802803 with bad export cookie 6436377631046614591 [16580.426064] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16580.436927] LustreError: 358873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [16584.677229] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16584.695272] Lustre: Skipped 2 previous similar messages [16587.227514] Lustre: server umount lustre-MDT0001 complete [16592.866369] Lustre: server umount lustre-OST0000 complete [16605.332136] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [16621.260581] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [16672.774420] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:21:33 (1785802893) [16682.000757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16682.404193] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [16682.466908] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [16682.637691] Lustre: lustre-MDT0000: new disk, initializing [16682.787353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16682.828536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [16690.530527] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [16709.867761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16710.203586] Lustre: 380406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16710.324867] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [16710.330735] Lustre: Skipped 1 previous similar message [16710.446920] Lustre: lustre-MDT0001: new disk, initializing [16710.700203] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16710.793232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [16710.817853] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [16718.059067] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [16730.695729] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16737.396514] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16747.165094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16747.654434] Lustre: lustre-OST0000: new disk, initializing [16747.669556] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [16747.687648] Lustre: 382378:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [16747.825119] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [16755.507416] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_default_debug -1 all [16757.406465] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [16757.429719] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [16757.532910] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [16768.475488] Lustre: DEBUG MARKER: oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16782.304957] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [16782.314923] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16782.333675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16782.336215] Lustre: Skipped 2 previous similar messages [16784.244503] Lustre: server umount lustre-MDT0000 complete [16788.462879] LustreError: 380414:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16788.490792] LustreError: 380414:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [16789.896345] LustreError: 380399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785803013 with bad export cookie 6436377631046616544 [16789.912668] LustreError: MGC192.168.204.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16790.653312] Lustre: server umount lustre-MDT0001 complete [16806.788796] Lustre: server umount lustre-OST0000 complete [16848.526372] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing set_hostid [16862.042442] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing load_modules_local [16910.792489] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 16536 sec ========= 20:25:30 (1785803130) [16913.525792] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:25:33 (1785803133) === [16918.947899] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:25:39 (1785803139) === [16981.858931] Lustre: DEBUG MARKER: oleg441-server.virtnet: executing unload_modules_local [16986.448639] Key type lgssc unregistered [16986.990530] LNet: 388757:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [16986.997252] LNetError: 388757:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [16987.025286] LNet: Removed LNI 192.168.204.141@tcp [16988.423303] Key type .llcrypt unregistered [16988.427367] Key type ._llcrypt unregistered