[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 594704640 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002333] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004013] kvm-guest: setup PV IPIs [ 0.007608] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008032] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009011] pid_max: default: 32768 minimum: 301 [ 0.011012] LSM: Security Framework initializing [ 0.012039] Yama: becoming mindful. [ 0.013028] SELinux: Initializing. [ 0.014067] *** VALIDATE selinux *** [ 0.022469] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027251] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028137] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030033] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031106] *** VALIDATE tmpfs *** [ 0.032428] *** VALIDATE proc *** [ 0.034134] *** VALIDATE cgroup *** [ 0.034921] *** VALIDATE cgroup2 *** [ 0.035239] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036131] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038024] Spectre V2 : User space: Vulnerable [ 0.039005] Speculative Store Bypass: Vulnerable [ 0.042560] debug: unmapping init [mem 0xffffffffb1259000-0xffffffffb1260fff] [ 0.044209] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045674] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046022] ... version: 2 [ 0.047009] ... bit width: 48 [ 0.048009] ... generic registers: 4 [ 0.049009] ... value mask: 0000ffffffffffff [ 0.050009] ... max period: 00007fffffffffff [ 0.051010] ... fixed-purpose events: 3 [ 0.052008] ... event mask: 000000070000000f [ 0.053331] rcu: Hierarchical SRCU implementation. [ 0.055427] smp: Bringing up secondary CPUs ... [ 0.056480] x86: Booting SMP configuration: [ 0.057016] .... node #0, CPUs: #1 #2 #3 [ 0.064023] smp: Brought up 1 node, 4 CPUs [ 0.066021] smpboot: Max logical packages: 1 [ 0.067011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.139495] node 0 deferred pages initialised in 70ms [ 0.142139] devtmpfs: initialized [ 0.144225] x86/mm: Memory block size: 128MB [ 0.146876] gcov: version magic: 0x41383552 [ 0.148187] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.151078] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.153302] pinctrl core: initialized pinctrl subsystem [ 0.155140] [ 0.155716] ************************************************************* [ 0.158012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.160008] ** ** [ 0.162015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.164015] ** ** [ 0.166017] ** This means that this kernel is built to expose internal ** [ 0.169011] ** IOMMU data structures, which may compromise security on ** [ 0.171010] ** your system. ** [ 0.173013] ** ** [ 0.175012] ** If you see this message and you are not debugging the ** [ 0.177010] ** kernel, report this immediately to your vendor! ** [ 0.180011] ** ** [ 0.182009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.184009] ************************************************************* [ 0.186910] NET: Registered protocol family 16 [ 0.188443] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.191058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.194049] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.198035] cpuidle: using governor menu [ 0.199454] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.201565] PCI: Using configuration type 1 for base access [ 0.203125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.214066] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.217019] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.222063] cryptd: max_cpu_qlen set to 1000 [ 0.225385] ACPI: Added _OSI(Module Device) [ 0.226021] ACPI: Added _OSI(Processor Device) [ 0.228013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.229018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.234633] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.240466] ACPI: Interpreter enabled [ 0.241000] ACPI: PM: (supports S0 S3 S4 S5) [ 0.243011] ACPI: Using IOAPIC for interrupt routing [ 0.245095] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.248790] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.258326] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.260030] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.262013] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.265086] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.269354] acpiphp: Slot [2] registered [ 0.270112] acpiphp: Slot [5] registered [ 0.272116] acpiphp: Slot [6] registered [ 0.273109] acpiphp: Slot [7] registered [ 0.275094] acpiphp: Slot [8] registered [ 0.276136] acpiphp: Slot [9] registered [ 0.278091] acpiphp: Slot [10] registered [ 0.279076] acpiphp: Slot [3] registered [ 0.280067] acpiphp: Slot [4] registered [ 0.282106] acpiphp: Slot [11] registered [ 0.283069] acpiphp: Slot [12] registered [ 0.285112] acpiphp: Slot [13] registered [ 0.286158] acpiphp: Slot [14] registered [ 0.288089] acpiphp: Slot [15] registered [ 0.289074] acpiphp: Slot [16] registered [ 0.290119] acpiphp: Slot [17] registered [ 0.291081] acpiphp: Slot [18] registered [ 0.292062] acpiphp: Slot [19] registered [ 0.293066] acpiphp: Slot [20] registered [ 0.294071] acpiphp: Slot [21] registered [ 0.295083] acpiphp: Slot [22] registered [ 0.296068] acpiphp: Slot [23] registered [ 0.297081] acpiphp: Slot [24] registered [ 0.298091] acpiphp: Slot [25] registered [ 0.299073] acpiphp: Slot [26] registered [ 0.301081] acpiphp: Slot [27] registered [ 0.302061] acpiphp: Slot [28] registered [ 0.303063] acpiphp: Slot [29] registered [ 0.304128] acpiphp: Slot [30] registered [ 0.305097] acpiphp: Slot [31] registered [ 0.306041] PCI host bridge to bus 0000:00 [ 0.307024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.309020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.311022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.313019] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.315021] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.318024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.319147] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.321864] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.325058] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.334015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.339463] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.341014] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.343015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.345014] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.347533] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.351437] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.354039] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.356775] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.361014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.373015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.377019] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.383338] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.425025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.434014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.525020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.573666] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.593015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.610016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.653019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.675855] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.708020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.731023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.769018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.795341] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.826022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.860016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.970023] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 1.010690] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.041019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.070015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.164019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.193011] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.221020] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.250018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.285019] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.306042] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.307310] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.309277] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.311271] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.313167] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.316066] iommu: Default domain type: Passthrough [ 1.318408] SCSI subsystem initialized [ 1.319170] ACPI: bus type USB registered [ 1.321143] usbcore: registered new interface driver usbfs [ 1.323096] usbcore: registered new interface driver hub [ 1.325115] usbcore: registered new device driver usb [ 1.327264] pps_core: LinuxPPS API ver. 1 registered [ 1.328006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.331079] PTP clock support registered [ 1.333085] EDAC MC: Ver: 3.0.0 [ 1.335180] PCI: Using ACPI for IRQ routing [ 1.337909] NetLabel: Initializing [ 1.339012] NetLabel: domain hash size = 128 [ 1.341017] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.343077] NetLabel: unlabeled traffic allowed by default [ 1.344343] vgaarb: loaded [ 1.346083] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.347008] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.354000] clocksource: Switched to clocksource kvm-clock [ 1.457008] VFS: Disk quotas dquot_6.6.0 [ 1.458257] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.460871] *** VALIDATE ramfs *** [ 1.462096] *** VALIDATE hugetlbfs *** [ 1.463503] pnp: PnP ACPI init [ 1.465865] pnp: PnP ACPI: found 6 devices [ 1.483465] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.486540] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.488837] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.490751] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.493078] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.494861] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.497667] NET: Registered protocol family 2 [ 1.500483] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.505560] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.509553] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.515906] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.519339] TCP: Hash tables configured (established 65536 bind 65536) [ 1.522274] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.525125] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.528598] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.532173] NET: Registered protocol family 1 [ 1.535502] RPC: Registered named UNIX socket transport module. [ 1.537837] RPC: Registered udp transport module. [ 1.539609] RPC: Registered tcp transport module. [ 1.541490] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.544155] NET: Registered protocol family 44 [ 1.545919] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.547464] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.549177] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.551552] PCI: CLS 0 bytes, default 64 [ 1.553230] Unpacking initramfs... [ 2.973744] debug: unmapping init [mem 0xffff93fcbcc54000-0xffff93fcbffbffff] [ 2.977795] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.980207] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.983054] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.497075] Initialise system trusted keyrings [ 3.498846] Key type blacklist registered [ 3.500753] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.510303] zbud: loaded [ 3.513425] *** VALIDATE nfs *** [ 3.514752] *** VALIDATE nfs4 *** [ 3.516386] pstore: using deflate compression [ 3.520421] Platform Keyring initialized [ 3.639181] NET: Registered protocol family 38 [ 3.641181] Key type asymmetric registered [ 3.642935] Asymmetric key parser 'x509' registered [ 3.645166] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.648710] io scheduler mq-deadline registered [ 3.650490] io scheduler kyber registered [ 3.652230] io scheduler bfq registered [ 3.654307] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.657408] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.660093] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.662978] ACPI: Power Button [PWRF] [ 3.668250] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.675189] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.686960] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.692807] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.710146] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.737758] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.767731] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.774161] Non-volatile memory driver v1.3 [ 3.775708] Linux agpgart interface v0.103 [ 3.805630] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.808130] vda: detected capacity change from 0 to 74702848 [ 3.821419] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.823798] vdb: detected capacity change from 0 to 1073741824 [ 3.835531] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.837968] vdc: detected capacity change from 0 to 2621440000 [ 3.850217] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.852564] vdd: detected capacity change from 0 to 2621440000 [ 3.863921] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.866238] vde: detected capacity change from 0 to 4294967296 [ 3.878849] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.881966] vdf: detected capacity change from 0 to 4294967296 [ 3.896549] libphy: Fixed MDIO Bus: probed [ 3.901499] usbcore: registered new interface driver usbserial_generic [ 3.903422] usbserial: USB Serial support registered for generic [ 3.905336] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.908849] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.910381] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.912520] mousedev: PS/2 mouse device common for all mice [ 3.915590] rtc_cmos 00:05: RTC can wake from S4 [ 3.919443] rtc_cmos 00:05: registered as rtc0 [ 3.921105] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.923546] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.923737] intel_pstate: CPU model not supported [ 3.929457] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.933640] hid: raw HID events driver (C) Jiri Kosina [ 3.935792] usbcore: registered new interface driver usbhid [ 3.937346] usbhid: USB HID core driver [ 3.938564] drop_monitor: Initializing network drop monitor service [ 3.940604] Initializing XFRM netlink socket [ 3.942407] NET: Registered protocol family 10 [ 3.942459] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.944881] Segment Routing with IPv6 [ 3.948097] NET: Registered protocol family 17 [ 3.950175] mpls_gso: MPLS GSO support [ 3.957339] RAS: Correctable Errors collector initialized. [ 3.958996] AVX version of gcm_enc/dec engaged. [ 3.960199] AES CTR mode by8 optimization enabled [ 4.053820] sched_clock: Marking stable (4053752395, 0)->(4904497290, -850744895) [ 4.056823] registered taskstats version 1 [ 4.058701] Loading compiled-in X.509 certificates [ 4.060778] zswap: loaded using pool lzo/zbud [ 4.088773] Key type big_key registered [ 4.101525] Key type encrypted registered [ 4.103098] ima: No TPM chip found, activating TPM-bypass! [ 4.104408] ima: Allocated hash algorithm: sha1 [ 4.106141] ima: No architecture policies found [ 4.107333] evm: Initialising EVM extended attributes: [ 4.108578] evm: security.selinux [ 4.109429] evm: security.ima [ 4.110208] evm: security.capability [ 4.111129] evm: HMAC attrs: 0x1 [ 4.113403] rtc_cmos 00:05: setting system clock to 2026-08-04 21:00:22 UTC (1785877222) [ 4.117904] debug: unmapping init [mem 0xffffffffb2203000-0xffffffffb23fffff] [ 4.119988] debug: unmapping init [mem 0xffffffffb0f82000-0xffffffffb1258fff] [ 4.132182] Write protecting the kernel read-only data: 28672k [ 4.134985] debug: unmapping init [mem 0xffffffffaf603000-0xffffffffaf7fffff] [ 4.137165] debug: unmapping init [mem 0xffffffffaff14000-0xffffffffafffffff] [ 4.170939] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.178275] systemd[1]: Detected virtualization kvm. [ 4.180311] systemd[1]: Detected architecture x86-64. [ 4.182352] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.207318] systemd[1]: No hostname configured. [ 4.208576] systemd[1]: Set hostname to . [ 4.210343] random: systemd: uninitialized urandom read (16 bytes read) [ 4.212191] systemd[1]: Initializing machine ID from random generator. [ 4.346226] random: systemd: uninitialized urandom read (16 bytes read) [ 4.348163] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.351806] random: systemd: uninitialized urandom read (16 bytes read) [ 4.353956] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 4.359794] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Listening on udev Control Socket. Starting Setup Virtual Console... [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Timers. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.010445] device-mapper: uevent: version 1.0.3 [ 5.012808] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.664803] virtio_net virtio0 ens2: renamed from eth0 [ 5.690083] random: fast init done [ 5.744774] scsi host0: ata_piix [ 5.755810] scsi host1: ata_piix [ 5.757425] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.760014] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.007890] dracut-initqueue[587]: RTNETLINK answers: File exists [ 10.628141] random: crng init done [ 10.629348] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.989535] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.094908] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.374525] SELinux: Disabled at runtime. [ 12.442075] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.451448] systemd[1]: Detected virtualization kvm. [ 12.453368] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.947532] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.951364] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.956516] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.959462] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.961821] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.968151] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.971569] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. Starting Apply Kernel Variables... [ 13.009564] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-getty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.444525] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.724556] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.781417] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.883644] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.895966] EDAC sbridge: Ver: 1.1.2 [ 15.661884] Key type dns_resolver registered [ 15.949151] NFS: Registering the id_resolver key type [ 15.951247] Key type id_resolver registered [ 15.952985] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. Starting Network Manager... Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg213-server login: [ 40.596743] libcfs: loading out-of-tree module taints kernel. [ 40.616308] Key type ._llcrypt registered [ 40.617941] Key type .llcrypt registered [ 40.664439] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 49.658167] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 50.386111] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 50.395740] alg: No test for adler32 (adler32-zlib) [ 51.415171] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 52.078845] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 53.928929] Key type lgssc registered [ 57.008167] Lustre: Echo OBD driver; http://www.lustre.org/ [ 77.368997] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 135.673568] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 144.805475] hrtimer: interrupt took 3558058 ns [ 155.891272] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 155.950247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 157.487976] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 157.573611] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 157.782872] Lustre: lustre-MDT0000: new disk, initializing [ 157.961594] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 157.987772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 165.100941] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.954163] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 185.130308] Lustre: 6517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 185.162109] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 185.168000] Lustre: Skipped 1 previous similar message [ 185.253866] Lustre: lustre-MDT0001: new disk, initializing [ 185.307749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 185.328287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 185.339465] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 192.216331] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 198.848642] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 215.028514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 215.406365] Lustre: lustre-OST0000: new disk, initializing [ 215.407872] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 215.413811] Lustre: 8458:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 215.479829] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 222.051439] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.858317] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 224.870662] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 225.004136] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 243.214245] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 243.442276] Lustre: lustre-OST0001: new disk, initializing [ 243.444459] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 243.460745] Lustre: 9532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 243.569763] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 250.953953] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 250.974590] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 251.077289] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 252.794082] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 268.600215] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 280.290481] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 288.997245] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing check_logdir /tmp/testlogs/ [ 297.094751] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing yml_node [ 303.127201] Lustre: DEBUG MARKER: Client: 2.17.56.50 [ 306.790752] Lustre: DEBUG MARKER: MDS: 2.17.56.50 [ 310.069782] Lustre: DEBUG MARKER: OSS: 2.17.56.50 [ 312.504437] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Aug 4 17:05:28 EDT 2026 [ 337.434062] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 339.188765] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 358.882080] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 358.885876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 358.902632] Lustre: Skipped 1 previous similar message [ 358.926235] Lustre: Skipped 1 previous similar message [ 364.258187] Lustre: server umount lustre-MDT0000 complete [ 369.122257] LustreError: 10159:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 369.147781] LustreError: 10159:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 369.829194] LustreError: 6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785877588 with bad export cookie 8714959940756142912 [ 369.834476] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 369.842529] LustreError: 6511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 374.243743] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 374.244022] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.248571] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 374.249291] Lustre: Skipped 2 previous similar messages [ 374.258127] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 374.338403] Lustre: Skipped 3 previous similar messages [ 376.651203] Lustre: server umount lustre-MDT0001 complete [ 383.649689] Lustre: server umount lustre-OST0000 complete [ 390.134715] Lustre: server umount lustre-OST0001 complete [ 401.045397] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 418.904592] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 475.626748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 476.000572] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 476.058168] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 476.218492] Lustre: lustre-MDT0000: new disk, initializing [ 476.322379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 476.347429] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 482.241781] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 497.318622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 497.433973] Lustre: 15116:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 497.505675] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 497.507333] Lustre: Skipped 1 previous similar message [ 497.543959] Lustre: lustre-MDT0001: new disk, initializing [ 497.645593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 497.700552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 497.720954] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 503.972918] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 521.467122] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 532.090141] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 542.204384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 542.838656] Lustre: lustre-OST0000: new disk, initializing [ 542.841323] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 542.845476] Lustre: 17089:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 542.986050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 544.329337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 544.335925] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 544.453805] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 550.443334] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 562.778838] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 568.888142] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 569.346367] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 576.547124] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 576.825202] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 580.583527] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 580.586736] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 580.587965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 580.587969] Lustre: Skipped 1 previous similar message [ 580.628309] Lustre: Skipped 1 previous similar message [ 580.687774] LustreError: Skipped 1 previous similar message [ 585.702800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 585.712152] Lustre: Skipped 2 previous similar messages [ 590.831034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 590.843587] Lustre: Skipped 1 previous similar message [ 594.400131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 594.630638] Lustre: server umount lustre-OST0000 complete [ 600.040455] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 600.053351] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 600.068698] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 606.178646] LustreError: 15127:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 606.189489] LustreError: 15127:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 606.345870] Lustre: server umount lustre-MDT0000 complete [ 611.326223] LustreError: 15126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 611.361526] LustreError: 15126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 614.000570] LustreError: 15106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785877832 with bad export cookie 8714959940756144445 [ 614.006966] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 614.022178] LustreError: 15106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 615.267762] Lustre: server umount lustre-MDT0001 complete [ 627.112167] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 17:10:42 (1785877842) [ 638.145075] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 639.215393] LustreError: 19360:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 639.328403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 646.250262] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 648.682165] LustreError: 19361:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 648.719912] LustreError: 19361:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 661.253902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 670.528287] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 685.880323] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 697.707471] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 709.935631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 710.769412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 710.777527] Lustre: Skipped 1 previous similar message [ 721.411328] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 732.761540] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 743.121422] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 754.152648] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 754.163703] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 754.193841] Lustre: Skipped 1 previous similar message [ 754.230613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 754.233312] Lustre: Skipped 2 previous similar messages [ 758.487280] Lustre: server umount lustre-MDT0000 complete [ 760.293383] LustreError: 19356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 760.327874] LustreError: 19356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 765.079821] LustreError: 19342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785877983 with bad export cookie 8714959940756145327 [ 765.090705] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 765.098118] LustreError: 19342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 765.411232] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 765.442769] Lustre: Skipped 2 previous similar messages [ 766.352816] Lustre: server umount lustre-MDT0001 complete [ 773.420800] Lustre: server umount lustre-OST0000 complete [ 806.193927] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 819.823425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 820.303566] LustreError: 23773:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 820.336780] LustreError: 23773:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 827.367712] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 839.861731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 840.588782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 840.591797] Lustre: Skipped 1 previous similar message [ 847.844771] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 852.298752] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 862.539647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 872.096064] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 887.669023] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 888.355189] Lustre: lustre-OST0001: new disk, initializing [ 888.362843] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 888.373969] Lustre: 26101:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 890.301430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 890.318636] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 890.673951] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 902.792036] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 916.673348] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 922.920482] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 941.545873] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 941.547428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 941.597093] Lustre: Skipped 2 previous similar messages [ 941.605897] Lustre: Skipped 5 previous similar messages [ 948.253352] Lustre: server umount lustre-MDT0000 complete [ 951.783446] LustreError: 23768:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 951.816736] LustreError: 23768:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 955.230494] LustreError: 23755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785878173 with bad export cookie 8714959940756146174 [ 955.240870] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 956.011115] Lustre: server umount lustre-MDT0001 complete [ 972.554863] Lustre: server umount lustre-OST0000 complete [ 973.152815] Lustre: 3634:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785878175/real 1785878175] req@ffff93fc06799180 x1872628044555648/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785878191 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 977.377177] Lustre: 3631:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785878179/real 1785878179] req@ffff93fc06798700 x1872628044555904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785878195 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 978.661792] Lustre: server umount lustre-OST0001 complete [ 1008.758619] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1026.799274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1033.675315] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1045.786510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1052.725200] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1056.662842] Lustre: 30281:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1065.620173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1075.147021] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1083.887609] LustreError: 30634:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1083.916110] LustreError: 30634:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 1086.883268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1096.002823] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1107.027910] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1112.707493] Lustre: 32162:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1126.487834] Lustre: server umount lustre-MDT0000 complete [ 1128.427139] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1128.429279] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1128.450778] Lustre: Skipped 2 previous similar messages [ 1132.006363] LustreError: 29112:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785878350 with bad export cookie 8714959940756147469 [ 1132.061663] LustreError: 29112:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1132.100448] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1133.307503] Lustre: server umount lustre-MDT0001 complete [ 1149.280142] Lustre: 3631:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785878351/real 1785878351] req@ffff93fd365db100 x1872628044614784/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785878367 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1149.305582] Lustre: 3631:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1151.820353] Lustre: server umount lustre-OST0000 complete [ 1153.381450] Lustre: 3633:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785878355/real 1785878355] req@ffff93fc059dc700 x1872628044615040/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785878371 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1157.621527] Lustre: server umount lustre-OST0001 complete [ 1170.043577] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 17:19:46 (1785878386) [ 1211.085964] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1227.286638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1227.909452] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1227.926468] Lustre: Skipped 6 previous similar messages [ 1234.839299] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1245.020496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1252.390927] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1256.031704] Lustre: 36130:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1266.023341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1275.475408] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1293.891649] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1308.211943] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1319.832780] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1325.062522] Lustre: 38011:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1333.227462] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1333.248232] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1333.263656] Lustre: Skipped 4 previous similar messages [ 1333.267760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1333.270483] Lustre: Skipped 6 previous similar messages [ 1340.391889] LustreError: 34977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1340.431330] LustreError: 34977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 22 previous similar messages [ 1341.468476] Lustre: server umount lustre-MDT0000 complete [ 1346.812906] LustreError: 34963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785878565 with bad export cookie 8714959940756148575 [ 1346.814745] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1346.820429] LustreError: 34963:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1347.405945] Lustre: server umount lustre-MDT0001 complete [ 1364.654514] Lustre: server umount lustre-OST0000 complete [ 1367.201387] Lustre: 3632:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785878569/real 1785878569] req@ffff93fd03e1dc00 x1872628044681216/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785878585 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1367.238313] Lustre: 3632:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1376.698227] Lustre: server umount lustre-OST0001 complete [ 1392.250725] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 17:23:28 (1785878608) [ 1415.094754] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1429.418424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1437.600193] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1450.880816] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1458.894285] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1462.952808] Lustre: 41310:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1473.165827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1481.977686] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1492.867367] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1502.389414] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1515.848164] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1522.205432] Lustre: 43187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1524.272244] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1524.291846] Lustre: Skipped 1 previous similar message [ 1529.319107] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1529.331348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1529.331926] Lustre: Skipped 6 previous similar messages [ 1529.344950] Lustre: Skipped 6 previous similar messages [ 1531.855567] Lustre: server umount lustre-MDT0000 complete [ 1538.204530] LustreError: 42497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785878756 with bad export cookie 8714959940756149681 [ 1538.206612] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1538.250769] LustreError: 42497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1544.717678] Lustre: server umount lustre-MDT0001 complete [ 1557.880566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1563.111972] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x78f1c1e46c879a57 [ 1563.151775] Lustre: MGC192.168.202.113@tcp: Connection restored to 0@lo (at 0@lo) [ 1570.163306] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1573.045587] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1586.777711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1593.404642] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1593.415851] Lustre: Skipped 1 previous similar message [ 1595.971887] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1609.948948] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1612.589142] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1620.871158] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1622.912575] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1633.783499] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1637.341427] Lustre: server umount lustre-MDT0000 complete [ 1642.915743] LustreError: 40137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785878861 with bad export cookie 8714959940756150871 [ 1642.940065] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1643.599893] Lustre: server umount lustre-MDT0001 complete [ 1659.652518] Lustre: server umount lustre-OST0000 complete [ 1660.384188] Lustre: 3634:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785878862/real 1785878862] req@ffff93fd379fd880 x1872628044792064/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785878878 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1660.405709] Lustre: 3634:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 1665.323389] Lustre: server umount lustre-OST0001 complete [ 1680.444909] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 17:28:15 (1785878895) [ 1702.142829] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1714.996891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1722.232933] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1734.343731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1741.339706] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1745.607847] Lustre: 48957:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1754.581966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1755.201371] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1755.212923] Lustre: Skipped 11 previous similar messages [ 1763.403714] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1774.660485] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1785.798825] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1797.149497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1804.143141] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1821.159124] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1821.167923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1821.187817] Lustre: Skipped 10 previous similar messages [ 1821.208255] Lustre: Skipped 14 previous similar messages [ 1825.552557] Lustre: server umount lustre-MDT0000 complete [ 1831.662673] LustreError: 47785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785879050 with bad export cookie 8714959940756151669 [ 1831.671958] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1831.675357] LustreError: 47785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1832.855825] Lustre: server umount lustre-MDT0001 complete [ 1849.523224] Lustre: server umount lustre-OST0000 complete [ 1851.873130] Lustre: 3634:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785879054/real 1785879054] req@ffff93fd39b6d500 x1872628044859776/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785879070 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1851.913163] Lustre: 3634:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1855.217953] Lustre: server umount lustre-OST0001 complete [ 1870.692721] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 17:31:25 (1785879085) [ 1874.440876] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1877.116232] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 17:31:32 (1785879092) [ 1892.509067] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 1904.605349] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1937.947274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1938.471214] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1938.493054] Lustre: Skipped 1 previous similar message [ 1938.582465] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1938.756790] Lustre: lustre-MDT0000: new disk, initializing [ 1939.080454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1945.363636] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1958.559788] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1969.016177] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1969.519561] Lustre: lustre-OST0000: new disk, initializing [ 1969.527212] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1969.534986] Lustre: Skipped 1 previous similar message [ 1969.548853] Lustre: 55041:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1971.513802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1971.528800] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1971.561649] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1978.225466] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1992.295147] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1995.720234] Lustre: DEBUG MARKER: create test files [ 2048.493431] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2050.073748] Lustre: server umount lustre-OST0000 complete [ 2056.235964] Lustre: server umount lustre-MDT0000 complete [ 2080.889541] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2087.385501] Key type lgssc unregistered [ 2087.922542] LNet: 56703:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2087.925339] LNetError: 56703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2087.987905] LNet: Removed LNI 192.168.202.113@tcp [ 2089.798229] Key type .llcrypt unregistered [ 2089.803265] Key type ._llcrypt unregistered [ 2129.718626] Key type ._llcrypt registered [ 2129.721184] Key type .llcrypt registered [ 2129.827147] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2131.625475] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2131.689059] alg: No test for adler32 (adler32-zlib) [ 2132.848871] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 2133.135547] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2134.800207] Key type lgssc registered [ 2136.164463] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2146.483705] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2146.511138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2147.900725] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2153.920429] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2167.348836] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2178.099735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2179.087677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2188.861199] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2193.400402] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2193.612082] Lustre: 59443:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 2193.648175] Lustre: 59443:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 2193.661929] Lustre: 59443:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2193.679315] Lustre: 59443:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2193.692548] Lustre: 59443:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2193.704809] Lustre: 59443:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2203.327204] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2209.844517] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2220.133688] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2237.807822] Lustre: server umount lustre-OST0000 complete [ 2256.254297] Lustre: server umount lustre-MDT0000 complete [ 2282.134500] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2286.896290] Key type lgssc unregistered [ 2287.436399] LNet: 61103:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2287.451276] LNetError: 61103:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2287.484967] LNet: Removed LNI 192.168.202.113@tcp [ 2288.961264] Key type .llcrypt unregistered [ 2288.964316] Key type ._llcrypt unregistered [ 2330.076056] Key type ._llcrypt registered [ 2330.088483] Key type .llcrypt registered [ 2330.361678] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2333.608112] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2333.711469] alg: No test for adler32 (adler32-zlib) [ 2335.348568] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 2335.864516] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2337.696179] Key type lgssc registered [ 2339.580549] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2351.049422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2351.066185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2352.592265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2358.249357] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2369.123259] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2378.297890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2378.794976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2387.722479] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2397.167237] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2397.277948] Lustre: 63944:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2397.295710] Lustre: 63944:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2397.303908] Lustre: 63944:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2397.311554] Lustre: 63944:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2397.326098] Lustre: 63944:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2397.336045] Lustre: 63944:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2400.286949] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2407.882139] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2422.755235] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2422.767285] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2422.778330] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2424.398673] Lustre: server umount lustre-OST0000 complete [ 2431.381406] Lustre: server umount lustre-MDT0000 complete [ 2450.433888] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2453.912736] Key type lgssc unregistered [ 2454.322494] LNet: 65549:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2454.339213] LNetError: 65549:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2454.367846] LNet: Removed LNI 192.168.202.113@tcp [ 2455.646311] Key type .llcrypt unregistered [ 2455.647781] Key type ._llcrypt unregistered [ 2481.719266] Key type ._llcrypt registered [ 2481.722100] Key type .llcrypt registered [ 2481.829661] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 2504.877492] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2507.213664] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2507.287353] alg: No test for adler32 (adler32-zlib) [ 2508.604734] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 2509.124260] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2510.944150] Key type lgssc registered [ 2513.022799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2562.834109] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2562.886140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2564.239569] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2564.312895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2564.489234] Lustre: lustre-MDT0000: new disk, initializing [ 2564.678500] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2564.705274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2570.505087] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2585.763955] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2586.006616] Lustre: 68709:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2586.147870] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2586.153595] Lustre: Skipped 1 previous similar message [ 2586.300277] Lustre: lustre-MDT0001: new disk, initializing [ 2586.445906] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2586.519675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2586.529028] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2593.399340] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2607.928512] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2616.267745] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2627.775402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2628.166698] Lustre: lustre-OST0000: new disk, initializing [ 2628.171230] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2628.176826] Lustre: 70681:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2628.292480] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2633.870794] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2633.883692] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2634.134519] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2640.120134] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2651.698689] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2658.715178] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2658.964518] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2665.603491] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2665.847605] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2670.051732] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2670.071265] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2670.091132] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2670.107515] Lustre: Skipped 1 previous similar message [ 2675.194026] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2675.203122] Lustre: Skipped 1 previous similar message [ 2680.294945] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2680.310950] Lustre: Skipped 1 previous similar message [ 2683.360378] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2683.760539] Lustre: server umount lustre-OST0000 complete [ 2694.114478] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2694.132226] LustreError: Skipped 1 previous similar message [ 2694.138342] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2694.169119] Lustre: Skipped 1 previous similar message [ 2694.181097] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2695.653815] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2697.234919] Lustre: server umount lustre-MDT0000 complete [ 2703.940582] LustreError: 68701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785879922 with bad export cookie 89635130143864768 [ 2703.945590] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2703.962543] LustreError: 68701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2704.018361] LustreError: 66930:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [ 2705.079734] Lustre: server umount lustre-MDT0001 complete [ 2718.328764] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 17:45:33 (1785879933) [ 2735.644815] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2792.824141] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 2807.987975] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2867.982138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2868.540457] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2868.618115] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2868.785753] Lustre: lustre-MDT0000: new disk, initializing [ 2868.870263] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2868.905370] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2875.248327] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2889.577611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2889.828918] Lustre: 76754:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2889.987071] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2889.994877] Lustre: Skipped 1 previous similar message [ 2890.142430] Lustre: lustre-MDT0001: new disk, initializing [ 2890.296469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2890.342727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2890.367160] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2898.203266] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2910.705481] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2920.322756] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2931.816214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2932.225416] Lustre: lustre-OST0000: new disk, initializing [ 2932.234298] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2932.241402] Lustre: 78723:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2932.430301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2941.098223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2941.125695] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2941.354158] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2942.498990] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2956.965734] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2964.563276] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2965.006512] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2972.326452] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2972.731798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2977.252939] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2977.256108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2977.268321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2977.268330] Lustre: Skipped 2 previous similar messages [ 2977.290842] Lustre: Skipped 1 previous similar message [ 2982.263383] Lustre: server umount lustre-OST0000 complete [ 2992.618808] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2992.631510] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2992.633366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2992.633372] Lustre: Skipped 1 previous similar message [ 2992.641128] Lustre: Skipped 1 previous similar message [ 2995.427284] Lustre: server umount lustre-MDT0000 complete [ 2996.870479] LustreError: 76765:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2996.931098] LustreError: 76765:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3001.737349] LustreError: 76744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785880220 with bad export cookie 89635130143865643 [ 3001.765646] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3001.767480] LustreError: 76744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3002.277885] Lustre: server umount lustre-MDT0001 complete [ 3016.080543] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 17:50:31 (1785880231) [ 3025.994309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3026.846527] LustreError: 81139:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3027.003806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3032.039797] LustreError: 81140:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3034.382581] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3037.162323] LustreError: 81139:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3042.277921] LustreError: 81140:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3047.314539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3054.250748] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3066.244628] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3075.516970] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3085.111333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3085.778887] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3085.784834] Lustre: Skipped 1 previous similar message [ 3096.129296] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3110.006746] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3117.735486] Lustre: 83818:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting a1786dc8-277e-46a1-ab65-cd13c5949520 at adminstrative request [ 3120.664476] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3120.772319] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 3131.503967] Lustre: MGS: Client a1786dc8-277e-46a1-ab65-cd13c5949520 (at 0@lo) reconnecting [ 3131.529506] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x13e72ad0b9efe9d to 0x13e72ad0b9f0144 [ 3131.557729] Lustre: MGC192.168.202.113@tcp: Connection restored to 0@lo (at 0@lo) [ 3160.501159] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3160.902970] Lustre: lustre-OST0001: new disk, initializing [ 3160.912513] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3160.925445] Lustre: 84211:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3161.097543] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3166.868474] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3166.876751] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3166.985302] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3172.510574] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3185.717709] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3192.800811] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3192.807484] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3192.821109] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3192.830070] Lustre: Skipped 1 previous similar message [ 3192.859220] Lustre: Skipped 2 previous similar messages [ 3196.189620] Lustre: server umount lustre-OST0001 complete [ 3197.931219] LustreError: 82968:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3197.948113] LustreError: 82968:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3201.507719] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3208.348877] Lustre: server umount lustre-OST0000 complete [ 3216.866706] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3216.884803] Lustre: Skipped 2 previous similar messages [ 3220.832354] Lustre: server umount lustre-MDT0000 complete [ 3223.523505] LustreError: 81139:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3223.562255] LustreError: 81139:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3226.673983] LustreError: 81119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785880445 with bad export cookie 89635130143867204 [ 3226.696745] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3226.729166] LustreError: 81119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3227.144572] Lustre: server umount lustre-MDT0001 complete [ 3240.465024] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 17:54:16 (1785880456) [ 3242.927729] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3245.305663] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 17:54:21 (1785880461) [ 3248.189444] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3251.570547] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 17:54:27 (1785880467) [ 3254.121883] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3257.108343] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 17:54:32 (1785880472) [ 3259.306558] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3262.133376] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 17:54:37 (1785880477) [ 3264.796628] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3273.749993] loop: module loaded [ 3273.791072] loop0: detected capacity change from 0 to 4294967296 [ 3273.888594] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3276.166732] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3281.459968] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3286.962520] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3328.694792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3328.746514] LustreError: 88340:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3328.758122] LustreError: 88340:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3328.776161] LustreError: 88340:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3328.792645] LustreError: 88340:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3358.266782] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 3377.541376] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3434.744443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3435.353160] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3435.409068] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3435.566172] Lustre: lustre-MDT0000: new disk, initializing [ 3435.726454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3435.766105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3442.091211] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3456.651414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3456.900561] Lustre: 91236:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3457.552806] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3463.687940] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3476.403440] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3484.339534] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3493.764461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3494.330718] Lustre: 93202:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3501.206878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3501.228750] Lustre: Skipped 1 previous similar message [ 3501.243193] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3501.364544] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3503.845302] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3517.226973] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3523.756364] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3524.090923] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3531.314668] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3531.625043] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3537.385203] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3537.392824] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3537.399410] LustreError: Skipped 1 previous similar message [ 3537.407912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3537.418133] Lustre: Skipped 2 previous similar messages [ 3537.447165] Lustre: Skipped 7 previous similar messages [ 3541.009063] Lustre: server umount lustre-OST0000 complete [ 3553.944652] Lustre: server umount lustre-MDT0000 complete [ 3556.840722] LustreError: 91247:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3556.871832] LustreError: 91247:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3559.492197] LustreError: 91228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785880777 with bad export cookie 89635130143867820 [ 3559.498300] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3559.521988] LustreError: 91228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3560.251478] Lustre: server umount lustre-MDT0001 complete [ 3573.326192] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 17:59:49 (1785880789) [ 3584.688463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3592.700425] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3608.797662] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3617.884325] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3629.899784] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3641.399663] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3650.652212] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3660.635473] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3671.663125] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3679.388386] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3679.838378] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3687.656945] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3688.015886] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3697.017169] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3697.280354] Lustre: lustre-OST0001: new disk, initializing [ 3697.285197] Lustre: Skipped 2 previous similar messages [ 3697.289462] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3697.295078] Lustre: Skipped 3 previous similar messages [ 3697.311744] Lustre: 99006:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3697.431903] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3697.434266] Lustre: Skipped 5 previous similar messages [ 3705.169645] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3705.499707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3705.517518] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3705.741837] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3718.296290] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3725.201832] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3725.603982] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3732.544601] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3733.106884] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3763.042972] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3803.545756] Lustre: Failing over lustre-MDT0000 [ 3804.134690] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3804.139335] LustreError: Skipped 1 previous similar message [ 3804.141593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3804.146670] Lustre: Skipped 2 previous similar messages [ 3804.150797] LustreError: 97057:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3804.160539] LustreError: 97057:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3804.221423] Lustre: server umount lustre-MDT0000 complete [ 3824.611710] Lustre: 66930:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785881026/real 1785881026] req@00000000a543ee1f x1872630620956800/t0(0) o400->MGC192.168.202.113@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785881042 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3824.652969] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3838.375502] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3838.387578] LDISKFS-fs (dm-0): recovery complete [ 3838.414714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3853.182194] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3854.282292] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3854.282831] Lustre: 99013:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3858.419838] Lustre: 95621:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3858.464772] Lustre: 95621:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 3858.526495] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3858.740671] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3859.097142] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3859.465561] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3859.505430] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 3859.529576] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3859.552118] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3859.896320] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3859.899713] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3860.592992] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3860.608518] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3860.942260] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3860.958958] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3861.643946] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3861.665272] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 5 previous similar messages [ 3862.617292] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3862.637443] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 3862.709826] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3862.981182] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3862.993446] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 3865.670522] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3865.687934] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 11 previous similar messages [ 3866.657581] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3866.665823] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 11 previous similar messages [ 3867.008122] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3867.031109] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 11 previous similar messages [ 3873.797975] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3873.839813] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 3874.854460] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3874.865034] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3875.201940] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3875.213890] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3889.971717] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3889.987727] Lustre: 101868:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 3890.957218] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3890.966741] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 3891.296776] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3891.312161] LustreError: 101868:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 3914.032139] Lustre: 101868:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3914.051652] Lustre: 101868:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3914.071706] Lustre: 101868:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 384241d7-4ce2-43ea-b75c-ebe823a880ca@192.168.202.13@tcp [ 3914.094429] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3914.099656] Lustre: 101868:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000c51ef325 x1872630596572800/t0(8589934771) o101->384241d7-4ce2-43ea-b75c-ebe823a880ca@192.168.202.13@tcp:368/0 lens 592/0 e 0 to 0 dl 1785881143 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3914.150264] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3914.166517] Lustre: lustre-MDT0000: Denying connection for new client 384241d7-4ce2-43ea-b75c-ebe823a880ca (at 192.168.202.13@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3914.184263] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3914.269476] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3914.269664] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3914.293295] Lustre: Skipped 2 previous similar messages [ 3914.313592] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 3914.326304] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3935.720723] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3935.747195] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3935.775545] Lustre: Skipped 3 previous similar messages [ 3935.785537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3935.797291] Lustre: Skipped 4 previous similar messages [ 3940.027138] Lustre: server umount lustre-OST0000 complete [ 3943.414259] LustreError: 97455:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3943.441505] LustreError: 97455:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 69 previous similar messages [ 3953.165740] Lustre: server umount lustre-OST0001 complete [ 3965.675548] Lustre: server umount lustre-MDT0000 complete [ 3971.705159] LustreError: 95601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785881190 with bad export cookie 89635130143919039 [ 3971.713897] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3971.721089] LustreError: 95601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3972.658731] Lustre: server umount lustre-MDT0001 complete [ 3985.099906] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:06:40 (1785881200) [ 3997.026289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3997.037753] Lustre: *** cfs_fail_loc=197, val=0*** [ 4012.705619] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4017.824769] LustreError: 103846:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.113@tcp: failed processing log, type 4: rc = -110 [ 4023.035097] Lustre: *** cfs_fail_loc=197, val=0*** [ 4023.039199] Lustre: Skipped 65 previous similar messages [ 4052.275870] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4066.526850] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4070.515846] Lustre: server umount lustre-OST0000 complete [ 4083.700628] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:08:19 (1785881299) [ 4108.366539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4115.466146] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4130.476369] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4135.813581] LustreError: 105500:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 4135.830685] LustreError: 105500:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 4135.844401] LustreError: 105500:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 27, retries 0, failed: rc = -5 [ 4136.454535] Lustre: server umount lustre-MDT0000 complete [ 4185.027789] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 4199.630846] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4260.439976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4260.898219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4260.959343] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4261.102794] Lustre: lustre-MDT0000: new disk, initializing [ 4261.196440] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4261.198779] Lustre: Skipped 3 previous similar messages [ 4261.222984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4266.846255] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4282.253330] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4282.397591] Lustre: 109680:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4282.604214] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4292.136465] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4311.050641] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4320.597993] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4331.667450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4332.307390] Lustre: 111651:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4339.505381] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4342.122655] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4355.275927] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4363.164571] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4363.574291] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4370.324638] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4370.597806] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4375.536850] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4375.541075] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4375.557129] Lustre: Skipped 5 previous similar messages [ 4375.564081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4375.564089] Lustre: Skipped 8 previous similar messages [ 4375.607925] LustreError: Skipped 2 previous similar messages [ 4387.297550] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4387.632744] Lustre: server umount lustre-OST0000 complete [ 4399.761408] Lustre: server umount lustre-MDT0000 complete [ 4401.121085] LustreError: 109692:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4401.144084] LustreError: 109692:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 4405.007565] LustreError: 109673:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785881623 with bad export cookie 89635130143929560 [ 4405.019837] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4405.029342] LustreError: 109673:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4405.759447] Lustre: server umount lustre-MDT0001 complete [ 4419.104354] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:13:54 (1785881634) [ 4433.957825] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 4439.621779] Key type lgssc unregistered [ 4440.094267] LNet: 114160:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4440.106243] LNetError: 114160:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4440.128744] LNet: Removed LNI 192.168.202.113@tcp [ 4441.636748] Key type .llcrypt unregistered [ 4441.640303] Key type ._llcrypt unregistered [ 4471.966721] Key type ._llcrypt registered [ 4471.968853] Key type .llcrypt registered [ 4472.105223] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 4499.966661] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4503.709837] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4503.901990] alg: No test for adler32 (adler32-zlib) [ 4505.233955] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 4505.650927] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 4507.322379] Key type lgssc registered [ 4509.559181] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4571.005949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4571.037402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4572.318543] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4572.376780] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4572.524407] Lustre: lustre-MDT0000: new disk, initializing [ 4572.680780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4572.728585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4579.382534] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4594.385595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4595.132269] Lustre: lustre-OST0000: new disk, initializing [ 4595.142610] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4595.156439] Lustre: Skipped 1 previous similar message [ 4595.181842] Lustre: 118344:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4595.364130] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4604.068527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4604.083894] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4604.128743] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4605.149908] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4619.734351] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4619.958570] Lustre: lustre-OST0001: new disk, initializing [ 4619.967455] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4619.980732] Lustre: 119209:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4620.117740] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4626.662673] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4626.668383] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4626.727083] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4627.699907] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4642.278497] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4642.299105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4646.972908] Lustre: server umount lustre-MDT0000 complete [ 4655.810030] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4662.322697] LustreError: 117529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785881880 with bad export cookie 1077952572096093993 [ 4662.332247] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4662.335817] LustreError: 117529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4662.443905] Lustre: server umount lustre-OST0000 complete [ 4668.191243] Lustre: server umount lustre-OST0001 complete [ 4681.013108] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 18:18:16 (1785881896) [ 4715.479404] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 18:18:50 (1785881930) [ 4730.099085] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4766.510188] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:19:42 (1785881982) [ 4823.388330] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 4838.780828] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4891.603201] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4892.147622] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4892.214887] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4892.409241] Lustre: lustre-MDT0000: new disk, initializing [ 4892.620480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4892.714742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4898.846473] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4913.179866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4913.391714] Lustre: 126996:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4913.490934] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4913.499186] Lustre: Skipped 1 previous similar message [ 4913.624364] Lustre: lustre-MDT0001: new disk, initializing [ 4913.794930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4913.875486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4913.895090] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4921.312763] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4939.971145] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4951.957543] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4962.889729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4963.445482] Lustre: lustre-OST0000: new disk, initializing [ 4963.454580] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4963.468620] Lustre: 128966:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4963.664841] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4972.688396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4972.708072] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4972.872616] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4975.124117] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4990.755081] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4997.852791] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4998.091146] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5003.892192] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5004.268191] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5008.871479] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5008.882645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5008.886102] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5008.896789] Lustre: Skipped 2 previous similar messages [ 5008.922362] Lustre: Skipped 2 previous similar messages [ 5013.436630] Lustre: server umount lustre-OST0000 complete [ 5019.109097] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5019.135196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5019.141677] Lustre: Skipped 1 previous similar message [ 5024.226316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5024.235257] Lustre: Skipped 1 previous similar message [ 5024.827670] Lustre: server umount lustre-MDT0000 complete [ 5030.077368] LustreError: 126988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785882248 with bad export cookie 1077952572096095218 [ 5030.088515] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5030.088988] LustreError: 126988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5030.914683] Lustre: server umount lustre-MDT0001 complete [ 5042.282986] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:24:18 (1785882258) [ 5052.006100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5052.953590] LustreError: 131381:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5052.977165] LustreError: 131381:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5053.145080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5058.531954] LustreError: 131382:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5058.553816] LustreError: 131382:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5060.362364] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5062.643567] LustreError: 131381:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5067.746310] LustreError: 131382:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5072.890910] LustreError: 131381:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5072.986320] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5080.787427] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5090.893602] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5100.482985] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5112.726285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5113.676535] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5113.685394] Lustre: Skipped 1 previous similar message [ 5124.425784] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5137.114195] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5143.382416] Lustre: *** cfs_fail_loc=159, val=0*** [ 5159.962803] Lustre: lustre-MDT0000: Client 74f5f6f9-9d77-4b2f-a5df-ec30ca0feabe (at 192.168.202.13@tcp) reconnecting [ 5162.280799] Lustre: *** cfs_fail_loc=159, val=0*** [ 5162.282398] Lustre: Skipped 5 previous similar messages [ 5178.913395] Lustre: lustre-MDT0000: Client 74f5f6f9-9d77-4b2f-a5df-ec30ca0feabe (at 192.168.202.13@tcp) reconnecting [ 5185.518747] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5185.527414] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5185.543918] LustreError: Skipped 1 previous similar message [ 5185.549591] Lustre: Skipped 1 previous similar message [ 5185.553824] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5185.592424] Lustre: Skipped 1 previous similar message [ 5192.242836] Lustre: server umount lustre-OST0000 complete [ 5200.872134] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5200.882224] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5200.899619] Lustre: Skipped 2 previous similar messages [ 5200.930376] Lustre: Skipped 3 previous similar messages [ 5203.819069] Lustre: server umount lustre-MDT0000 complete [ 5205.990398] LustreError: 132112:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5206.018607] LustreError: 132112:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5208.962706] LustreError: 134108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785882427 with bad export cookie 1077952572096096100 [ 5208.965123] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5208.975835] LustreError: 134108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5209.525445] Lustre: server umount lustre-MDT0001 complete [ 5228.420685] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 5233.540632] Key type lgssc unregistered [ 5234.073603] LNet: 135186:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5234.116321] LNetError: 135186:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5234.143944] LNet: Removed LNI 192.168.202.113@tcp [ 5235.613308] Key type .llcrypt unregistered [ 5235.616147] Key type ._llcrypt unregistered [ 5269.057989] Key type ._llcrypt registered [ 5269.059926] Key type .llcrypt registered [ 5269.192197] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 5270.811132] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5271.046327] alg: No test for adler32 (adler32-zlib) [ 5272.225764] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 5272.510047] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 5274.232450] Key type lgssc registered [ 5276.163304] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5287.462653] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5287.510496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5289.073347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5296.024566] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5298.663200] LustreError: 136604:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5298.679628] LustreError: 136604:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5303.778341] LustreError: 136603:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5308.586041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5309.023712] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5316.358100] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5327.294286] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5337.669835] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5350.473844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5351.112745] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5360.372846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 5361.310367] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5374.920474] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5386.501794] Lustre: *** cfs_fail_loc=159, val=0*** [ 5403.163922] Lustre: lustre-MDT0000: Client f9cd0761-3f82-442f-a05b-878e0a6dadcf (at 192.168.202.13@tcp) reconnecting [ 5407.523482] Lustre: *** cfs_fail_loc=159, val=0*** [ 5424.270449] Lustre: lustre-MDT0001: Client f9cd0761-3f82-442f-a05b-878e0a6dadcf (at 192.168.202.13@tcp) reconnecting [ 5427.614569] Lustre: *** cfs_fail_loc=159, val=0*** [ 5427.621943] Lustre: Skipped 4 previous similar messages [ 5444.129778] Lustre: lustre-MDT0001: Client f9cd0761-3f82-442f-a05b-878e0a6dadcf (at 192.168.202.13@tcp) reconnecting [ 5451.769664] Lustre: 139648:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5455.685130] Lustre: *** cfs_fail_loc=159, val=0*** [ 5455.694986] Lustre: Skipped 3 previous similar messages [ 5471.284250] Lustre: lustre-MDT0000: Client 7c1fb6a8-fa1c-46a8-a494-da8baba45e51 (at 192.168.202.13@tcp) reconnecting [ 5474.205552] Lustre: *** cfs_fail_loc=159, val=0*** [ 5474.212151] Lustre: Skipped 14 previous similar messages [ 5490.687664] Lustre: lustre-MDT0000: Client 7c1fb6a8-fa1c-46a8-a494-da8baba45e51 (at 192.168.202.13@tcp) reconnecting [ 5493.531180] Lustre: 139940:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5513.185857] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5513.197889] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5513.217481] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5514.215224] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5514.221600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5514.240097] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5514.293422] Lustre: Skipped 1 previous similar message [ 5518.807255] Lustre: server umount lustre-OST0000 complete [ 5523.942218] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5523.962497] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5524.003397] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5529.579964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5529.593294] Lustre: Skipped 3 previous similar messages [ 5534.697263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5534.711588] Lustre: Skipped 1 previous similar message [ 5538.272636] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5538.785587] LustreError: 136603:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5538.811245] LustreError: 136603:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5538.832757] Lustre: server umount lustre-MDT0000 complete [ 5543.910202] LustreError: 139304:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5543.943886] LustreError: 139304:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5545.106824] LustreError: 136584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785882763 with bad export cookie 14703314180494420388 [ 5545.117039] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5545.129451] LustreError: 136584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5546.133379] Lustre: server umount lustre-MDT0001 complete [ 5563.764542] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 5568.405514] Key type lgssc unregistered [ 5568.789579] LNet: 141025:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5568.807715] LNetError: 141025:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5568.832710] LNet: Removed LNI 192.168.202.113@tcp [ 5570.665588] Key type .llcrypt unregistered [ 5570.668135] Key type ._llcrypt unregistered [ 5602.537138] Key type ._llcrypt registered [ 5602.539694] Key type .llcrypt registered [ 5602.767372] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 5605.101659] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5605.171367] alg: No test for adler32 (adler32-zlib) [ 5606.378587] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 5606.717126] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 5608.571106] Key type lgssc registered [ 5611.412255] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5626.976368] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5626.987985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5628.759944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5634.971865] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5639.138296] LustreError: 142445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5639.170747] LustreError: 142445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5644.264014] LustreError: 142444:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5649.379204] LustreError: 142445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5651.297379] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5651.991592] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5661.858163] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5673.390330] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5681.064570] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5690.989202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5691.676424] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5696.827470] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5698.900325] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5701.833722] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5712.041969] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5720.224777] Lustre: 145028:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5726.322557] Lustre: 145077:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5728.413594] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5729.500987] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5735.436131] Lustre: 145223:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5736.925314] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5738.105660] Lustre: 145332:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5745.122188] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5745.136858] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5745.169817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5748.193262] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5748.205185] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5748.247421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5751.254223] Lustre: server umount lustre-OST0000 complete [ 5759.459141] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5759.470329] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5759.493703] Lustre: Skipped 1 previous similar message [ 5759.512449] Lustre: Skipped 3 previous similar messages [ 5762.914757] Lustre: server umount lustre-MDT0000 complete [ 5764.582868] LustreError: 142445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5764.640765] LustreError: 142445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5768.608413] LustreError: 142425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785882986 with bad export cookie 7683594366205346663 [ 5768.622598] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5769.208842] Lustre: server umount lustre-MDT0001 complete [ 5789.696869] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 5795.244349] Key type lgssc unregistered [ 5795.708419] LNet: 146418:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5795.723812] LNetError: 146418:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5795.749360] LNet: Removed LNI 192.168.202.113@tcp [ 5797.294532] Key type .llcrypt unregistered [ 5797.299684] Key type ._llcrypt unregistered [ 5831.752246] Key type ._llcrypt registered [ 5831.759706] Key type .llcrypt registered [ 5831.965593] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 5834.313935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5834.356483] alg: No test for adler32 (adler32-zlib) [ 5835.704832] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 5836.154374] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 5837.968196] Key type lgssc registered [ 5840.341857] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5857.166372] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5857.186866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5858.868897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5865.288539] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5869.031131] LustreError: 147835:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5869.056409] LustreError: 147835:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5874.151236] LustreError: 147836:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5879.266315] LustreError: 147835:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5879.298803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5880.122610] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5888.463942] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5901.880685] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5910.623618] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5920.990295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5921.509385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5925.625435] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5928.711180] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5932.273783] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5945.378613] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5953.624123] Lustre: *** cfs_fail_loc=159, val=0*** [ 5969.391433] Lustre: lustre-MDT0000: Client ad3632fb-7334-4dfc-88f5-c9167522b3a5 (at 192.168.202.13@tcp) reconnecting [ 5990.371536] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5990.375495] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5990.398719] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5990.414025] Lustre: Skipped 1 previous similar message [ 5990.422403] LustreError: 150454:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5990.824830] Lustre: server umount lustre-OST0000 complete [ 5998.049174] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5998.061637] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5998.080358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6000.609590] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6000.613267] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6000.658175] Lustre: Skipped 1 previous similar message [ 6002.657357] Lustre: server umount lustre-MDT0000 complete [ 6005.745681] LustreError: 150450:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6005.777117] LustreError: 150450:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6008.099807] LustreError: 147815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785883226 with bad export cookie 3699146816726984166 [ 6008.111900] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6008.116154] LustreError: 147815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6009.031608] Lustre: server umount lustre-MDT0001 complete [ 6026.561333] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 6033.970307] Key type lgssc unregistered [ 6035.795484] LNet: 151513:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6035.823604] LNetError: 151513:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6036.844735] LNet: Removed LNI 192.168.202.113@tcp [ 6038.835672] Key type .llcrypt unregistered [ 6038.837698] Key type ._llcrypt unregistered [ 6075.497441] Key type ._llcrypt registered [ 6075.501581] Key type .llcrypt registered [ 6075.685355] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 6077.272126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6077.530764] alg: No test for adler32 (adler32-zlib) [ 6079.139652] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 6079.603740] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 6081.456323] Key type lgssc registered [ 6083.739246] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6096.540211] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6096.572099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6098.817629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6107.211407] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6109.167795] LustreError: 152934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6109.204264] LustreError: 152934:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6114.275313] LustreError: 152933:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6119.395182] LustreError: 152934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6119.854353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6120.465026] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6127.064301] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6138.223274] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6146.719200] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6159.762880] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6160.475613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6165.566344] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 6169.691671] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 6171.854356] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6186.786317] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6212.685335] Lustre: lustre-OST0000: evicting beb9c902-c1c4-4a4e-8015-13d2b7b4dde5 (at 192.168.202.13@tcp) by administrative request [ 6212.696492] Lustre: Skipped 1 previous similar message [ 6251.493764] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6251.501853] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6251.536900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6254.343908] Lustre: server umount lustre-OST0000 complete [ 6261.731068] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6261.734693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6261.754753] Lustre: Skipped 2 previous similar messages [ 6261.792109] Lustre: Skipped 2 previous similar messages [ 6266.857344] LustreError: 153680:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6267.170674] Lustre: server umount lustre-MDT0000 complete [ 6271.978433] LustreError: 152934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6272.016577] LustreError: 152934:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6273.323159] LustreError: 153675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785883491 with bad export cookie 13409002694390625163 [ 6273.333492] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6273.345407] LustreError: 153675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6274.158822] Lustre: server umount lustre-MDT0001 complete [ 6293.426620] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 6297.864230] Key type lgssc unregistered [ 6298.370395] LNet: 157036:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6298.382032] LNetError: 157036:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6299.431421] LNet: Removed LNI 192.168.202.113@tcp [ 6301.376936] Key type .llcrypt unregistered [ 6301.380581] Key type ._llcrypt unregistered [ 6415.669955] Key type ._llcrypt registered [ 6415.672287] Key type .llcrypt registered [ 6415.755659] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 6417.290371] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6417.541881] alg: No test for adler32 (adler32-zlib) [ 6418.991291] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 6419.556642] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 6421.498537] Key type lgssc registered [ 6423.738675] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6436.339134] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6436.376388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6438.314375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6445.712108] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6448.624303] LustreError: 159548:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6448.648718] LustreError: 159548:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6453.734176] LustreError: 159547:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6457.491634] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6458.147115] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6464.500108] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6475.903961] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6485.006966] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6494.947493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6495.588994] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6502.800077] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 6508.987048] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 6510.239265] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6524.043968] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6534.678385] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6534.678587] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6534.718509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6534.744167] Lustre: Skipped 1 previous similar message [ 6539.744946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6541.062559] Lustre: server umount lustre-OST0000 complete [ 6549.986355] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6550.003904] Lustre: Skipped 1 previous similar message [ 6550.016811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6550.030174] Lustre: Skipped 2 previous similar messages [ 6554.298221] Lustre: server umount lustre-MDT0000 complete [ 6555.105938] LustreError: 159548:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6555.151300] LustreError: 159548:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6560.227927] LustreError: 160480:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6560.272722] LustreError: 160480:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6561.224407] LustreError: 160291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785883779 with bad export cookie 5471034420954578208 [ 6561.233984] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6561.242745] LustreError: 160291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6562.178336] Lustre: server umount lustre-MDT0001 complete [ 6583.190540] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 6587.634466] Key type lgssc unregistered [ 6588.189776] LNet: 163120:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6588.214982] LNetError: 163120:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6588.247933] LNet: Removed LNI 192.168.202.113@tcp [ 6589.662573] Key type .llcrypt unregistered [ 6589.665142] Key type ._llcrypt unregistered [ 6628.065418] Key type ._llcrypt registered [ 6628.070698] Key type .llcrypt registered [ 6628.248346] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 6630.396044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6630.479809] alg: No test for adler32 (adler32-zlib) [ 6631.875621] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 6632.379452] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 6634.240195] Key type lgssc registered [ 6636.105796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6657.239292] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:51:13 (1785883873) [ 6681.735328] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 6700.164827] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 6750.663406] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6750.708332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6752.383339] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6752.475648] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6752.700359] Lustre: lustre-MDT0000: new disk, initializing [ 6752.857487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6752.883534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6759.951958] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6775.855202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6776.042305] Lustre: 167410:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6776.114827] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6776.123502] Lustre: Skipped 1 previous similar message [ 6776.251540] Lustre: lustre-MDT0001: new disk, initializing [ 6776.410621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6776.470682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6776.502530] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6782.573980] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6795.299845] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6804.716943] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6815.402310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6815.755317] Lustre: lustre-OST0000: new disk, initializing [ 6815.758550] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6815.762576] Lustre: 169383:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6815.912632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6822.879494] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6824.089658] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6824.103955] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6824.340582] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6836.660815] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6855.141225] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6855.154142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6855.168337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6855.197807] Lustre: Skipped 1 previous similar message [ 6855.226579] Lustre: Skipped 1 previous similar message [ 6857.438682] Lustre: server umount lustre-OST0000 complete [ 6865.391666] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6865.398264] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6865.419582] Lustre: Skipped 1 previous similar message [ 6869.966484] Lustre: server umount lustre-MDT0000 complete [ 6875.230825] LustreError: 167403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785884093 with bad export cookie 10965949662359124741 [ 6875.241186] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6875.244166] LustreError: 167403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6876.425799] Lustre: server umount lustre-MDT0001 complete [ 6893.571501] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 6897.260299] Key type lgssc unregistered [ 6897.782493] LNet: 171291:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6897.794616] LNetError: 171291:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6898.856846] LNet: Removed LNI 192.168.202.113@tcp [ 6899.860761] Key type .llcrypt unregistered [ 6899.863206] Key type ._llcrypt unregistered [ 6947.103565] Key type ._llcrypt registered [ 6947.104858] Key type .llcrypt registered [ 6947.201973] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 6949.589970] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6949.658258] alg: No test for adler32 (adler32-zlib) [ 6951.137468] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 6951.637899] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 6953.472160] Key type lgssc registered [ 6955.676764] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6968.022383] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6968.075810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6969.751125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6977.261441] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 6980.091163] LustreError: 172710:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6980.114094] LustreError: 172710:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6985.195347] LustreError: 172709:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6990.307823] LustreError: 172710:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6992.320839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6993.121507] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7001.537702] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 7014.661555] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7022.846845] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7033.428401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7033.876203] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7040.092089] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7043.514548] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 7055.624024] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7065.634587] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7065.650597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7065.661274] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7065.665831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7065.682820] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7066.843576] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7066.880778] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7066.888709] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7066.903482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7066.911976] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7068.197597] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7068.243207] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7068.260390] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7068.269945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7068.293744] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7069.912807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7071.520295] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7071.537111] Lustre: Skipped 1 previous similar message [ 7071.557795] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7071.575144] Lustre: Skipped 1 previous similar message [ 7071.592421] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7071.601872] Lustre: Skipped 1 previous similar message [ 7071.609432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7071.626293] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7071.629620] Lustre: Skipped 1 previous similar message [ 7072.998086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7074.391852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.759741] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7075.771535] Lustre: Skipped 2 previous similar messages [ 7075.798455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7075.810161] Lustre: Skipped 2 previous similar messages [ 7075.827665] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7075.836198] Lustre: Skipped 2 previous similar messages [ 7075.839720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.855687] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7075.865037] Lustre: Skipped 2 previous similar messages [ 7077.028816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.446520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.064614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7081.284753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7082.636186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7083.776885] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7083.780808] Lustre: Skipped 5 previous similar messages [ 7083.800788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7083.818776] Lustre: Skipped 5 previous similar messages [ 7083.824115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7085.139683] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7085.147893] Lustre: Skipped 6 previous similar messages [ 7085.164060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7085.207587] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7085.217785] Lustre: Skipped 6 previous similar messages [ 7086.473638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.807058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7089.136319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7090.264571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.390141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7092.887125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7094.467116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7095.922466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7097.579622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.267761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7100.779899] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7100.786334] Lustre: Skipped 11 previous similar messages [ 7100.823945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7100.842965] Lustre: Skipped 11 previous similar messages [ 7100.860777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7102.430381] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7102.435397] Lustre: Skipped 11 previous similar messages [ 7102.440624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7102.461994] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7102.473103] Lustre: Skipped 11 previous similar messages [ 7103.908085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.026584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7107.735545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7109.264644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7111.176620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7113.176688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7115.247773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7117.201621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.529283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.048770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7121.478855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.097354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.312311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7125.992473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.073388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.706849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.990453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7131.554745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7132.844356] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7132.853886] Lustre: Skipped 19 previous similar messages [ 7132.898646] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7132.914959] Lustre: Skipped 19 previous similar messages [ 7132.920153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7134.288249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.469948] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7135.474573] Lustre: Skipped 20 previous similar messages [ 7135.478264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.487182] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7135.497392] Lustre: Skipped 20 previous similar messages [ 7136.482955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.749180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.147138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.715476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7142.668045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.457122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7146.086097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7147.837738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7149.439530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7150.834162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7152.172035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.733095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7155.336420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.085500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7158.485203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7159.540737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7161.040603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7162.421647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7163.402414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.311989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7165.398034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7166.290174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.859612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7169.248782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.118648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7172.122386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7173.259896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7174.013635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7175.025520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7176.777907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7178.615906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7179.822339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7181.244268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.221848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7183.526538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7184.732994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7185.734769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7187.117995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7188.671082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7190.122424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7191.949922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.295689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7194.716316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7196.304664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7197.620022] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7197.633662] Lustre: Skipped 46 previous similar messages [ 7197.671596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7197.688287] Lustre: Skipped 46 previous similar messages [ 7197.701165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7199.246691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.859673] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7200.862207] Lustre: Skipped 46 previous similar messages [ 7200.866718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.887442] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7200.899170] Lustre: Skipped 46 previous similar messages [ 7202.001968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7203.203473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7204.222606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7205.458994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.461520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7207.948887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7209.558207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7211.056321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7212.712412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7214.009541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7215.339783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7216.962504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7219.285050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7220.953383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7221.878038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7223.146310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7224.573317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7226.354676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.619088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.588562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7230.721189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.983586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7233.495372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.847941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7235.870599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7236.939434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.178483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7239.563435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7241.225506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7242.678227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.355260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7245.064514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.007528] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7246.012694] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7246.023479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.345928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7248.767948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7250.367021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7252.208873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7253.996061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7255.599684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7258.140166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7259.352244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7260.639371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.687488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7264.994242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7267.123967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.832872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7272.032705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7274.451177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7276.434545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.943974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7278.989553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7280.721490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7282.597682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7284.975895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7287.439701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.681586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7290.589137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7292.784564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7294.233564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7295.957105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7298.148000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7301.996102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7304.545108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.685241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7308.972039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7311.066461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7312.559814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.903208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7315.072692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7316.551605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7317.967401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7319.242717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7320.360894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7321.776115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.133343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7324.678949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7326.017838] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7326.028661] Lustre: Skipped 78 previous similar messages [ 7326.065958] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7326.082595] Lustre: Skipped 78 previous similar messages [ 7326.093660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.402556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7328.668474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.925208] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7329.936390] Lustre: Skipped 79 previous similar messages [ 7329.944036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.973759] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7329.987818] Lustre: Skipped 79 previous similar messages [ 7331.573322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7333.018514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.737643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7336.345563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.563840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7338.916637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7340.024107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.174582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.041934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7343.335444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7344.674701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7346.246326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7347.613036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7349.079276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7351.082650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.549166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.097287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7355.932720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7357.854259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7359.356093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7360.138630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7361.085775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7362.716392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7363.953694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.963086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.205939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7367.043358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7367.967844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7369.556161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.585567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7371.785691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7373.380560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7374.447556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7375.457466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7376.775903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.405240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7379.673246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7381.017946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7382.561712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7384.234293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.625402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7387.114544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7388.338596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7389.642050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7390.933649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.273414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7393.658163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7394.929607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7395.865648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7397.054584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7398.413865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.386828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7400.692149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7402.183435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7403.503285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7404.844341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7406.813424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.179359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7409.458344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7410.908466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7412.607821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7414.517342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.020750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.854636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7420.076668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.888588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7423.484064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.858212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.384739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.139728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.985671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7431.354117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7432.670263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7434.159230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7435.426300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.840492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7437.836664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7439.013393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7440.059043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.258975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7442.708290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7444.256430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.675644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.106513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7448.324731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7449.885767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7451.609909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7452.797726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7453.717507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7455.182203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7456.583633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7458.081759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7459.545038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7461.009285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7462.801574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7464.185133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7466.003311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7467.032319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7468.680696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7470.456777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7471.998423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7473.667327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7475.164321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.595413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7477.933113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7479.648727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.279267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7483.057274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7484.821075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.190484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7487.902722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7489.466256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7490.839933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.059239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.284796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7494.662936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.893478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7497.136414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7498.410425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7499.884906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7501.545787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7502.953391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7504.533393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.001863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7507.452658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.799281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7510.305118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7511.851487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7513.202051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7514.606951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7515.764820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7516.666435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7517.764375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7518.904816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7520.045041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7521.289046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7522.511790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7523.719538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7525.140226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7526.452231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7527.831467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7529.410474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7530.883663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.286802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7533.614481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7534.827944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7536.210833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.837492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7539.333404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.842332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7542.512689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7544.038253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7545.351980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7546.966435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7548.477248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7550.106246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7551.670205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7553.046712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7554.747807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.183817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7557.698564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7558.982871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7560.418639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7561.942453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7563.216813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7564.553701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7566.063647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7567.570209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7568.971985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7570.706739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7572.035909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7573.624569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7575.064627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7576.413852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.883203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7579.187470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7580.930285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.299058] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7582.305639] Lustre: Skipped 180 previous similar messages [ 7582.328570] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7582.336865] Lustre: Skipped 180 previous similar messages [ 7582.341078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7583.516624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7584.541520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.020957] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7586.033707] Lustre: Skipped 180 previous similar messages [ 7586.050689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.066289] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7586.073211] Lustre: Skipped 180 previous similar messages [ 7587.599318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7589.283401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7590.788819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7592.141460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7593.489044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7595.030423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7596.393911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7597.646806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7598.884229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7600.135793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7601.347478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7602.783268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.027966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7605.410024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7606.632255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7608.053633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7609.603287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7611.125243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7612.470256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7613.739953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7615.353163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7616.911917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7618.242758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7619.647516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7621.530542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7623.496588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7625.485213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7627.108827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7628.294264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7629.872426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7630.943360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7631.833242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7633.158075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7634.217836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7635.699737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7636.967690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7638.229608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7639.285804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7640.779337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7642.218874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7643.505164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7644.427407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7645.271909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.550895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7647.981531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7649.252574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7650.762937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7652.126360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7653.111103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7654.576132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7656.013685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7657.505392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7659.033946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7660.424945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7662.104941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7663.533758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7664.717240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7665.840202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7666.963198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7668.179755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7669.314855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7670.702490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7671.973424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7673.212745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7674.544118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7676.296449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7677.795886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.582040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7679.826470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7681.054249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7682.288444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7683.462091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7684.555470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7685.901472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7687.383983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7688.694732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7690.044683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7691.187190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7692.772938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7694.348257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7695.833795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7697.387952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7699.049765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7700.716773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7702.436069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7703.981606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7705.467144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7706.815843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7708.129527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7709.552461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7711.246380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7712.401911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7713.784040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7715.215812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.624440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7718.062546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7719.479452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7720.743274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7722.266660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7723.778661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7725.347664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7726.835044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7728.641499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7730.118607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7731.227945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7732.606260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7734.258310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7735.413733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7736.871812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7738.455561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7739.871525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7741.235342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7742.956972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7744.888879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7746.122157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7747.553404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7749.270991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7751.340125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7753.548786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7755.314173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7757.271622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7759.723894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7761.840501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7764.448356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7766.643052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7768.634947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7770.040878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7771.726436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7773.222983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7774.833193] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7774.841829] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7774.872275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7776.281435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7777.920767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7779.505801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7781.446859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7783.626447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7785.422269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7787.189626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7788.815228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7790.463407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7792.167952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7793.590869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7795.196461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7796.624483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7798.293076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7799.816511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7800.935826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7802.345800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7804.241763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7805.489791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7806.857420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7808.216663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7809.654450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7811.259720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7812.450653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7813.989092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7814.957471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7816.515393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.093306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7819.558774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7821.031904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7822.948754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7824.840653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7826.653813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7828.046082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7830.245286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7832.853774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7835.038991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7837.753355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7841.172697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7843.916923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7846.279459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7848.447440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7850.164675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7851.563595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7853.399235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7854.794583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7856.369118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7857.780191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7859.258154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7860.480678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7861.983644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7863.429814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7864.920900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7866.476475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7868.005568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7869.513740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7870.868868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7871.891648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7873.247280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7874.752197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7876.130068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.386250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7878.834026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7880.179395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.698074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7883.169339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7884.460101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7885.658790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7886.754836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7888.170918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7889.629992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.234379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7892.646462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7894.042191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7895.716733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7897.382510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7898.615106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7899.936847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7901.465050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7903.006625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7904.543160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7905.621584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7906.994050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7908.486024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7910.120849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7911.553514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7912.944148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7914.960892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7916.403342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7917.734463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7919.153876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7920.822050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7922.334602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7924.069388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7925.664749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7926.936057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7928.434698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7929.871636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7931.287224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7932.800650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7934.333544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7935.408985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7936.589537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7938.024139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7939.918661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7941.688426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.960846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7943.936183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7945.913989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7947.109708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7949.069824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7950.929256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7951.940644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7953.330973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7954.411786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7955.425810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7956.775961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7958.106485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7959.476338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7961.344332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7963.570106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7965.034503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7966.153938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7967.454085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7968.453582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7969.735059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7971.427832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7972.918558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7974.873481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7976.149607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7977.697417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7979.143189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7980.806636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7982.156389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7983.596657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7985.072694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7986.594276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7988.037569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7989.430749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7990.925302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7992.478391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7993.723027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7995.199729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7996.821793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7998.114129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7999.332837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8000.487210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8001.698827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8003.219140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8004.628875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8005.821708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8007.010246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8008.237418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8009.432246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8010.787182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8012.392522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8013.903702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8015.178180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8016.810759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8018.370819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8020.050195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8021.491163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8022.913453] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8022.925901] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8022.964985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8024.441850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8025.841643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8027.560599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8029.123023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8030.646453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8032.255338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8033.825902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8035.121833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8036.387682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8037.547595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8038.527344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8039.738820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8040.730802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8042.130893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8043.010165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8044.136305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8045.527569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8046.914440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8048.095379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8049.500791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8051.060620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8052.639112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8053.847682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8055.406373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8056.597619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8058.037652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8059.431689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8061.525602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8062.876443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8064.493307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8066.243800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8067.965713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8070.044973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8071.738968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8073.393608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8075.150702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8076.340202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8078.887305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8079.854791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8080.814539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8082.292746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8084.120327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8085.705283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8087.918268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8089.458754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8091.213139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8092.687805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8093.633891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8095.619845] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8095.632250] Lustre: Skipped 344 previous similar messages [ 8095.676186] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8095.701281] Lustre: Skipped 344 previous similar messages [ 8095.714932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8097.067520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8098.306230] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8098.312429] Lustre: Skipped 343 previous similar messages [ 8098.317322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8098.330263] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8098.348281] Lustre: Skipped 343 previous similar messages [ 8099.727562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8101.113120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8102.706700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8103.701472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8105.074316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8106.299277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8107.684644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8108.807890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8109.672034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8111.282071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8113.330128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8114.789408] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8114.803293] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8114.813600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8116.140429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8117.634767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8119.097161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8120.144525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8121.453030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8122.757620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8124.072653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8125.449060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8127.014517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8128.704683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8130.475890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8132.075029] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8132.108351] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8132.147842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8133.785688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8135.537849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8136.732762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8138.131367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8139.813956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8141.581088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8143.489878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8145.363351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8147.278681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8149.160454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8150.582226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8152.021815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8153.374948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8155.312964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8156.402575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8157.917824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8159.273805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8160.583954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8161.956590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8163.400519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8164.888488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8166.305952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8167.682376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8169.187871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8170.309566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8171.693635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8173.075600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8174.287091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8175.976746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8177.608926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8179.075650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8180.370836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8182.024563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8183.041889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8184.545833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8185.724917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8186.545517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8187.867437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8189.412965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8191.121958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8192.933503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8193.973682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8195.434078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8197.280911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8200.156639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8203.371019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8205.196771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8207.566838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8209.594630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8212.428337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8214.456957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8215.737846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8216.868808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8218.415351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8219.943459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8221.222815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8222.540299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8223.842521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8225.153145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8226.602767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8228.096564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8229.667158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8231.109210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8232.561294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8234.101342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8235.430836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8236.992145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8238.149454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8239.459459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8240.997193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8242.526226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8244.075651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8245.585228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8247.155476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8248.561898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8249.840276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8251.663839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8253.287474] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8253.308853] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8253.323309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8255.119788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8256.876400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8258.833287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8260.363355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8261.559777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8263.334169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8265.272539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8266.913936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8267.886671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8268.737399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8269.699527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8271.023571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8272.040927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8273.530345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8274.780224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8276.553303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8278.275656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8279.668662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8280.917491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8282.370494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8283.886579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8285.332569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8286.768708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8288.366450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8290.016629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8291.513642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8292.530242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8293.738553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8295.161838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8296.539771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8297.852456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8299.117447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8300.352033] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8300.363980] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8300.386032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8301.642341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8303.044916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8304.362743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8306.026371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8307.664452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8309.022398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8310.386105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8311.997186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8313.341400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8314.591659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8315.697152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8317.063293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8318.335750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8319.654805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8321.232534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8322.655591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8323.960142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8325.087968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8326.151066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8327.104447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8328.452714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8329.639981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8330.637540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8332.066205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8333.538802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8334.895663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8336.299781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8337.692943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8339.097799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8340.716804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8342.105485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8343.383348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8344.759065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8346.152467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8347.493671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8348.900249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8350.222538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8351.774629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8353.210575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8354.404191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8355.774775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8357.082513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8358.487031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8359.933038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8361.504922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8363.321387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8364.670804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8366.306710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8368.189291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8369.932846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8371.511456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8373.086524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8374.354021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8375.952381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8377.624484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8379.752130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8381.845271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8383.437400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8384.863741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8386.830667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8388.215041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8389.503153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8390.876909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8392.689931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8394.725297] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8394.757298] LustreError: 172729:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8394.778501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8397.103729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8399.197641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8400.870573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8402.316576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8403.846678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8405.007143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8406.338241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8407.495082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8408.637619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8409.854128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8411.069216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8412.416928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8413.913872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8415.043237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8416.411935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8417.966160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8419.416099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8421.027798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8422.346648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8423.478176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8424.906235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8425.970082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8427.143336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8428.568644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8429.856331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8431.113251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8432.214214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8433.899657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8435.473535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8436.673587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8438.190276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8440.135289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8441.813667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8443.315909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8445.249873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8447.186969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8449.245814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8451.184778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8453.147084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8454.905236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8456.668698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8458.073928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8459.391823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8460.764710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8462.117594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8463.372057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8465.268501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8466.733656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8467.876755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8469.523486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8471.214630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8472.679249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8474.241178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8475.672651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8477.259765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8478.818310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8480.370491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8481.939710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8483.448943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8484.952396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8486.792584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8487.949657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8488.662177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8489.797465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8491.077158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8492.488262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8493.746696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8494.989834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8496.536025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8498.128072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8499.345535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8500.967708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8502.231607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8503.396719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8504.735233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8506.055639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8507.550339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8508.978220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8510.066960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8511.092686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8512.242224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8513.486508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8514.724906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8516.088460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8517.195888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8518.220057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8519.414027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8520.581212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8521.807283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8522.965984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8523.993533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8525.430804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8526.719800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8528.034364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8529.576456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8530.628469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8531.894502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8533.186875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8534.205296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8535.246881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9146.038212] LustreError: 174536:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 9146.052693] LustreError: 172729:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [10120.162415] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10120.174336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10120.190072] Lustre: Skipped 303 previous similar messages [10120.210531] Lustre: Skipped 1 previous similar message [10125.285711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10125.290718] Lustre: Skipped 1 previous similar message [10126.263551] Lustre: server umount lustre-OST0000 complete [10135.523575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10135.537390] Lustre: Skipped 1 previous similar message [10140.641688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10140.650190] Lustre: Skipped 1 previous similar message [10145.760845] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10145.765421] Lustre: Skipped 1 previous similar message [10146.272140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10146.714201] Lustre: server umount lustre-MDT0000 complete [10149.861680] LustreError: 172709:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10149.870400] LustreError: 172709:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10150.919972] LustreError: 223344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785887369 with bad export cookie 762885390586603710 [10150.926131] LustreError: 223344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10150.932741] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10151.623241] Lustre: server umount lustre-MDT0001 complete [10168.742572] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [10172.943512] Key type lgssc unregistered [10173.341232] LNet: 228165:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10173.364276] LNetError: 228165:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10173.394505] LNet: Removed LNI 192.168.202.113@tcp [10175.089805] Key type .llcrypt unregistered [10175.098154] Key type ._llcrypt unregistered [10212.255482] Key type ._llcrypt registered [10212.257019] Key type .llcrypt registered [10212.410983] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [10214.045366] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10214.092733] alg: No test for adler32 (adler32-zlib) [10215.272991] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [10215.560814] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [10217.248189] Key type lgssc registered [10219.419173] Lustre: Echo OBD driver; http://www.lustre.org/ [10230.855875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10230.868985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10232.294459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10237.312884] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10242.537659] LustreError: 229581:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10242.564730] LustreError: 229581:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10247.652079] LustreError: 229582:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10248.306310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10248.877615] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10255.868562] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10267.312934] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10276.615900] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10285.883619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10286.264522] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10296.523346] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10302.707488] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [10308.787036] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10315.288606] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [10317.003051] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10317.010293] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [10317.015201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [10317.028482] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [10317.033322] LustreError: 229601:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [10320.353896] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10320.369644] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10320.399642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10323.433257] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10323.449057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10323.483867] Lustre: Skipped 2 previous similar messages [10325.971426] Lustre: lustre-OST0000: Not available for connect from 192.168.202.13@tcp (stopping) [10328.557278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10328.569313] Lustre: Skipped 1 previous similar message [10333.671511] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10333.686376] Lustre: Skipped 2 previous similar messages [10334.176230] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10334.501837] Lustre: server umount lustre-OST0000 complete [10337.094101] LustreError: 229601:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [10383.701806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10384.266135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10389.468370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [10389.488793] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [10389.508821] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [10389.533798] Lustre: Skipped 1 previous similar message [10395.181407] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10409.425158] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10411.317511] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in IDLE state after 0 sec [10418.103490] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94f4c70b3800.ost_server_uuid 50 [10420.902280] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94f4c70b3800.ost_server_uuid in IDLE state after 0 sec [10430.433755] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10430.450606] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10430.482344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10430.497351] Lustre: Skipped 1 previous similar message [10430.520878] Lustre: Skipped 1 previous similar message [10433.493799] Lustre: server umount lustre-OST0000 complete [10440.674061] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10440.711971] Lustre: Skipped 1 previous similar message [10444.994617] Lustre: server umount lustre-MDT0000 complete [10445.805518] LustreError: 229576:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10445.827316] LustreError: 229576:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10449.889829] LustreError: 230310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10449.937445] LustreError: 230310:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10451.043380] LustreError: 229562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785887669 with bad export cookie 11032841781127240916 [10451.051472] LustreError: 229562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10451.052976] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10451.618402] Lustre: server umount lustre-MDT0001 complete [10468.092526] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [10472.765943] Key type lgssc unregistered [10473.419629] LNet: 234803:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10473.437160] LNetError: 234803:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10473.475213] LNet: Removed LNI 192.168.202.113@tcp [10474.820317] Key type .llcrypt unregistered [10474.822128] Key type ._llcrypt unregistered [10519.888503] Key type ._llcrypt registered [10519.890331] Key type .llcrypt registered [10519.955912] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [10521.139083] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10521.251847] alg: No test for adler32 (adler32-zlib) [10522.759922] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [10523.211882] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [10525.025150] Key type lgssc registered [10527.148464] Lustre: Echo OBD driver; http://www.lustre.org/ [10544.268228] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10544.317829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10546.185703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10552.939592] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10556.393087] LustreError: 236540:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10556.420412] LustreError: 236540:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10561.518169] LustreError: 236541:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10565.328301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10565.755910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10570.332477] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10576.021610] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10586.838476] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10587.489764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10591.642335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [10597.206283] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10609.357512] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10609.788920] Lustre: lustre-OST0001: new disk, initializing [10609.799482] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10609.808357] Lustre: 238849:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10610.048391] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10616.199392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10616.216847] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10616.459480] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10621.311178] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10636.268604] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10649.146101] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10672.633402] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10672.643389] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10672.649321] Lustre: Skipped 2 previous similar messages [10672.670204] Lustre: Skipped 2 previous similar messages [10676.220157] Lustre: server umount lustre-MDT0000 complete [10677.736670] LustreError: 238047:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10677.763255] LustreError: 238047:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10681.975733] LustreError: 236522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785887900 with bad export cookie 7542876113228054409 [10681.993701] LustreError: 236522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10681.995847] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10682.439012] Lustre: server umount lustre-MDT0001 complete [10698.966127] Lustre: server umount lustre-OST0000 complete [10699.040119] Lustre: 235803:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785887901/real 1785887901] req@ffff93fc070bc700 x1872639024018304/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785887917 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10699.084617] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10699.104102] Lustre: Skipped 1 previous similar message [10703.072297] Lustre: 235801:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785887905/real 1785887905] req@ffff93fc28274000 x1872639024018560/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785887921 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10704.136315] Lustre: server umount lustre-OST0001 complete [10734.485884] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [10751.878288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10752.470499] LustreError: 241964:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10752.552563] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10756.960706] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10757.605705] LustreError: 241965:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10766.827430] LustreError: 241965:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10766.853904] LustreError: 241965:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10768.679586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10769.152105] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10775.458956] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10779.932930] Lustre: 243106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10791.793338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10792.664567] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10794.731463] LustreError: 243463:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10794.793920] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [10796.862832] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [10803.438554] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10815.337030] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10821.144310] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [10824.664616] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10835.000988] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10840.409266] Lustre: 244981:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10855.913558] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10855.928474] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10855.964121] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10855.966365] Lustre: Skipped 1 previous similar message [10861.313469] Lustre: server umount mylustre-MDT0000 complete [10862.068564] LustreError: 243607:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10862.102507] LustreError: 243607:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [10867.383199] LustreError: 243108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785888085 with bad export cookie 7542876113228057272 [10867.392781] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10867.408515] LustreError: 243108:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10867.740281] Lustre: server umount mylustre-MDT0001 complete [10883.832303] Lustre: server umount mylustre-OST0000 complete [10887.650239] Lustre: 235804:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785888090/real 1785888090] req@ffff93fc272c0700 x1872639024091520/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785888106 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10887.708933] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10887.735513] Lustre: Skipped 3 previous similar messages [10889.445995] Lustre: server umount mylustre-OST0001 complete [10919.346241] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [10938.076674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10938.717727] LustreError: 247305:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10938.739611] LustreError: 247305:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [10938.851726] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10938.863084] Lustre: Skipped 1 previous similar message [10945.875525] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10961.192900] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10968.205293] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [10972.666773] Lustre: 248449:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10984.262625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10987.274093] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [10995.194902] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11009.070102] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [11010.627152] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11010.900119] Lustre: tfs-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11010.914731] Lustre: Skipped 2 previous similar messages [11016.191107] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [11021.458727] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11033.954714] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11039.817700] Lustre: 250325:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11052.005194] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11052.008095] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [11052.040138] Lustre: Skipped 2 previous similar messages [11052.051883] Lustre: Skipped 6 previous similar messages [11055.017256] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [11055.032390] Lustre: Skipped 3 previous similar messages [11057.708552] Lustre: server umount tfs-MDT0000 complete [11064.435441] LustreError: 247280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785888282 with bad export cookie 7542876113228059442 [11064.457031] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11064.466522] LustreError: 247280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11065.340258] Lustre: tfs-MDT0001-lwp-OST0000: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11065.361891] Lustre: Skipped 2 previous similar messages [11065.871372] Lustre: server umount tfs-MDT0001 complete [11070.120692] Lustre: server umount tfs-OST0000 complete [11074.885628] Lustre: server umount tfs-OST0001 complete [11108.643237] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [11125.701656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11126.684435] LustreError: 252579:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11126.717554] LustreError: 252579:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [11134.359572] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11145.668806] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11146.157381] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11146.167843] Lustre: Skipped 1 previous similar message [11152.549463] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11156.778711] Lustre: 253724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11168.836135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11171.702931] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [11171.721869] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [11179.434335] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11194.626072] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11199.995276] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [11202.256811] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11216.020653] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11221.787619] Lustre: 255598:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11231.126877] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:07:26 (1785888446) [11253.729377] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11253.748560] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11253.761035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11258.280409] Lustre: server umount lustre-MDT0000 complete [11263.198813] LustreError: 252555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785888481 with bad export cookie 7542876113228061360 [11263.206234] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11263.223065] LustreError: 252555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11266.537954] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11266.552250] Lustre: Skipped 5 previous similar messages [11270.413820] Lustre: server umount lustre-MDT0001 complete [11277.867827] Lustre: server umount lustre-OST0000 complete [11284.312898] Lustre: server umount lustre-OST0001 complete [11293.957609] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [11306.072598] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [11360.771113] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [11360.771113] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [11360.771113] [11360.823199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [11361.363364] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11361.452848] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11361.508185] Lustre: lustre-MDT0000: new disk, initializing [11361.594985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11367.897879] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11380.833216] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [11380.833216] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [11380.833216] [11380.857271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [11380.982786] Lustre: 258785:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11381.074333] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11381.082180] Lustre: Skipped 1 previous similar message [11381.264839] Lustre: lustre-MDT0001: new disk, initializing [11381.516098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11381.535607] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11388.502245] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11403.252950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11403.532214] Lustre: lustre-OST0000: new disk, initializing [11403.535265] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11403.540383] Lustre: 260368:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11403.622522] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11403.632322] Lustre: Skipped 4 previous similar messages [11405.378433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11405.411699] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11405.627100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11413.429265] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11431.509667] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11431.783741] Lustre: lustre-OST0001: new disk, initializing [11431.794890] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11431.822413] Lustre: 261241:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11433.785318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11433.804881] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11434.008450] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11441.774856] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11456.408540] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:11:12 (1785888672) [11483.752393] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:11:38 (1785888698) [11503.073540] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11503.087505] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11503.108507] Lustre: Skipped 5 previous similar messages [11503.126142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11503.130978] Lustre: Skipped 3 previous similar messages [11516.387039] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11516.975733] Lustre: server umount lustre-OST0000 complete [11521.003513] LustreError: 260352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11521.044467] LustreError: 260352:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 22 previous similar messages [11524.585789] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11524.588387] LustreError: Skipped 1 previous similar message [11529.908723] Lustre: server umount lustre-MDT0000 complete [11534.741639] LustreError: 258779:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785888753 with bad export cookie 7542876113228063145 [11534.755130] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11534.758791] LustreError: 258779:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11536.360078] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11536.371604] Lustre: Skipped 10 previous similar messages [11549.152129] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11549.581327] Lustre: server umount lustre-MDT0001 complete [11578.895258] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [11656.966492] Lustre: server umount lustre-OST0001 complete [11668.969498] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [11688.418819] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [11745.872160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11746.347318] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11746.402940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11746.555724] Lustre: lustre-MDT0000: new disk, initializing [11746.753019] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11753.772245] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11767.909466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11768.138877] Lustre: 266103:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11768.242653] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11768.249068] Lustre: Skipped 1 previous similar message [11768.380607] Lustre: lustre-MDT0001: new disk, initializing [11768.613374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11768.629287] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11775.033889] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11788.658697] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11796.036412] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11806.365248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11807.044615] Lustre: lustre-OST0000: new disk, initializing [11807.056953] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11807.076208] Lustre: 268074:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11808.521781] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11808.550029] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11808.853157] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11819.249940] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11835.242310] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11844.582239] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11844.583279] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11844.585569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11844.585693] Lustre: Skipped 3 previous similar messages [11844.590497] LustreError: Skipped 1 previous similar message [11844.623392] Lustre: Skipped 6 previous similar messages [11850.640388] Lustre: server umount lustre-OST0000 complete [11855.854592] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11870.176142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11870.409563] Lustre: server umount lustre-MDT0000 complete [11876.783150] LustreError: 266899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785889095 with bad export cookie 7542876113228064615 [11876.799855] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11876.808711] LustreError: 266899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11878.177273] Lustre: server umount lustre-MDT0001 complete [11897.780859] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [11904.425876] Key type lgssc unregistered [11905.059442] LNet: 269843:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11905.080102] LNetError: 269843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11905.109299] LNet: Removed LNI 192.168.202.113@tcp [11906.819160] Key type .llcrypt unregistered [11906.824496] Key type ._llcrypt unregistered [11942.758476] Key type ._llcrypt registered [11942.761166] Key type .llcrypt registered [11942.890976] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [11944.325982] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11944.517351] alg: No test for adler32 (adler32-zlib) [11945.800918] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [11946.256217] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [11948.040241] Key type lgssc registered [11950.470259] Lustre: Echo OBD driver; http://www.lustre.org/ [11963.898617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11963.942990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11965.959833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11972.442658] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [11976.163993] LustreError: 271309:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11976.199214] LustreError: 271309:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11981.283603] LustreError: 271310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11984.085175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11984.513353] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11990.900539] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12002.760249] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12011.310604] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12021.728367] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12021.951684] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12031.300691] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12036.344021] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [12044.141503] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12045.495581] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [12049.896147] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12049.904211] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12049.914959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12051.937417] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12051.939596] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12051.955254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12052.000220] Lustre: Skipped 1 previous similar message [12055.204614] Lustre: server umount lustre-OST0000 complete [12062.182765] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12062.203834] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12067.312800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12067.340404] Lustre: Skipped 2 previous similar messages [12067.856677] Lustre: server umount lustre-MDT0000 complete [12072.421689] LustreError: 272931:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12072.448737] LustreError: 272931:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12073.782048] LustreError: 273844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785889292 with bad export cookie 397943674913602694 [12073.802510] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12073.803083] LustreError: 273844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12074.334536] Lustre: server umount lustre-MDT0001 complete [12093.854334] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [12098.150459] Key type lgssc unregistered [12098.602655] LNet: 274926:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12098.610546] LNetError: 274926:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12098.636683] LNet: Removed LNI 192.168.202.113@tcp [12100.081226] Key type .llcrypt unregistered [12100.085898] Key type ._llcrypt unregistered [12125.648805] Key type ._llcrypt registered [12125.652684] Key type .llcrypt registered [12125.775695] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [12127.623916] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12128.020997] alg: No test for adler32 (adler32-zlib) [12129.851942] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [12130.314337] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [12132.296159] Key type lgssc registered [12135.005738] Lustre: Echo OBD driver; http://www.lustre.org/ [12149.222075] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12149.265532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12164.960526] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [12170.080342] LustreError: 276121:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.113@tcp: failed processing log, type 4: rc = -110 [12196.832194] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12208.753507] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12221.203117] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12224.951451] Lustre: server umount lustre-OST0000 complete [12253.462779] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [12259.738369] Key type lgssc unregistered [12260.439174] LNet: 277772:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12260.451458] LNetError: 277772:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12260.475737] LNet: Removed LNI 192.168.202.113@tcp [12261.621275] Key type .llcrypt unregistered [12261.625274] Key type ._llcrypt unregistered [12323.448817] Key type ._llcrypt registered [12323.461178] Key type .llcrypt registered [12323.776605] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [12327.083257] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12327.133147] alg: No test for adler32 (adler32-zlib) [12328.987965] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [12329.890880] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [12331.984300] Key type lgssc registered [12334.188085] Lustre: Echo OBD driver; http://www.lustre.org/ [12355.347450] loop0: detected capacity change from 0 to 204800000 [12355.599041] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12355.631656] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12356.497982] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12358.067574] loop0: detected capacity change from 0 to 204800000 [12358.393481] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12358.428275] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12359.279218] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12361.170791] loop0: detected capacity change from 0 to 204800000 [12361.498721] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12361.504978] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12361.578618] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12362.613848] loop0: detected capacity change from 0 to 204800000 [12362.783017] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12362.803061] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12362.910108] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12364.167983] loop0: detected capacity change from 0 to 204800000 [12364.216306] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12370.634106] loop0: detected capacity change from 0 to 204800000 [12370.667617] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12376.424957] loop0: detected capacity change from 0 to 204800000 [12376.535899] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12383.275981] loop0: detected capacity change from 0 to 204800000 [12383.334399] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12389.651346] loop0: detected capacity change from 0 to 204800000 [12389.847476] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12389.936982] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12390.138689] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12390.154891] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [12390.180482] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [12390.195242] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [12390.210941] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [12390.220964] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [12390.249552] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [12390.264820] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [12390.282433] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [12390.291150] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [12390.298940] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [12390.314362] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [12390.322292] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [12390.342538] Lustre: 281196:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [12398.751286] Lustre: server umount MGS complete [12400.311534] loop0: detected capacity change from 0 to 204800000 [12400.438815] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12401.744358] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12401.873770] Lustre: lustre-MDT0000: Aborting client recovery [12403.430551] loop1: detected capacity change from 0 to 204800000 [12403.667512] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12403.936194] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12404.013292] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [12404.049171] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [12404.087768] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [12404.106962] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [12404.143247] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [12404.171665] Lustre: 281630:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [12404.230376] Lustre: Found index 1 for lustre-MDT0001, updating log [12404.803584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12404.921603] Lustre: lustre-MDT0001: Aborting client recovery [12406.556614] loop2: detected capacity change from 0 to 204800000 [12406.804645] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12406.950917] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12406.979770] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [12407.024385] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [12407.066397] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [12407.109479] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [12407.143426] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [12407.170461] Lustre: 281711:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [12407.372722] Lustre: Found index 0 for lustre-OST0000, updating log [12407.442458] Lustre: 281729:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12407.558714] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12407.626256] Lustre: lustre-OST0000: Aborting recovery [12409.274031] loop3: detected capacity change from 0 to 204800000 [12409.525798] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12409.617631] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12409.635852] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [12409.648315] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [12409.657124] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [12409.664828] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [12409.673773] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [12409.681656] Lustre: 281798:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [12409.699135] Lustre: Found index 1 for lustre-OST0001, updating log [12409.729462] Lustre: 281806:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12409.800906] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12409.814330] Lustre: lustre-OST0001: Aborting recovery [12409.862269] LustreError: 281574:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [12411.304759] Lustre: lustre-MDT0000: trigger OI scrub by RPC for [0x200000402:0x1:0x0]/0 with flags 0x4a: rc = 0 [12411.820522] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [12411.844939] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [12411.928889] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [12412.020739] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [12414.898245] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [12414.908828] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [12420.078023] LustreError: 281541:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1086us: rc = -2 [12420.591214] LustreError: 281538:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2523us: rc = -2 [12420.627014] LustreError: 281538:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 25 previous similar messages [12435.431480] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12435.436393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12435.456361] Lustre: Skipped 2 previous similar messages [12435.479563] Lustre: Skipped 1 previous similar message [12435.480241] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12440.556094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12440.558450] Lustre: Skipped 2 previous similar messages [12442.666500] Lustre: server umount lustre-MDT0000 complete [12444.500835] LustreError: 281525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785889662 with bad export cookie 17053025557820726325 [12444.507352] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12444.520883] LustreError: 281525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12445.666113] LustreError: 281538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12445.668546] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12445.688413] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12445.741681] LustreError: 281538:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12445.766296] Lustre: Skipped 2 previous similar messages [12449.697239] LustreError: 281992:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12449.697466] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12449.806963] Lustre: Skipped 2 previous similar messages [12451.910922] Lustre: server umount lustre-MDT0001 complete [12459.978648] Lustre: server umount lustre-OST0000 complete [12476.384669] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12476.698871] Lustre: server umount lustre-OST0001 complete [12490.531142] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:28:25 (1785889705) [12535.064291] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [12549.253532] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [12607.536263] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12608.060753] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12608.121287] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12608.265436] Lustre: lustre-MDT0000: new disk, initializing [12608.407340] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12608.430956] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12614.026490] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12628.377281] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12628.580109] Lustre: 285896:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12628.683324] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12628.690153] Lustre: Skipped 1 previous similar message [12628.737517] Lustre: lustre-MDT0001: new disk, initializing [12628.900667] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12628.984594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12629.003136] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12635.092167] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12648.759405] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12657.494968] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12667.357595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12667.883300] Lustre: lustre-OST0000: new disk, initializing [12667.890292] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12667.902725] Lustre: 287865:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12668.053359] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12673.629938] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12673.657286] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12673.898229] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12676.985846] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12689.590028] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12697.493151] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12709.621570] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12717.376553] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12726.536774] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [12776.420674] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12776.423323] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12776.489756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12776.504237] Lustre: Skipped 1 previous similar message [12781.358790] Lustre: server umount lustre-OST0000 complete [12791.782774] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12791.791176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12791.814492] Lustre: Skipped 2 previous similar messages [12791.843190] Lustre: Skipped 1 previous similar message [12794.996065] Lustre: server umount lustre-MDT0000 complete [12795.952221] LustreError: 285907:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12795.977733] LustreError: 285907:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12800.993568] LustreError: 289479:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12801.004960] LustreError: 289479:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12801.389881] LustreError: 289067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785890019 with bad export cookie 17053025557820728964 [12801.411539] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12801.412225] LustreError: 289067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12802.386738] Lustre: server umount lustre-MDT0001 complete [12811.623726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12817.509659] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12823.773196] Lustre: server umount MGS complete [12838.930204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12839.550428] LustreError: 291689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12839.581152] LustreError: 291702:0:(qmt_pool.c:1615:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [12839.633183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12845.343217] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12849.128850] LustreError: 291690:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12849.164509] LustreError: 291690:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12855.586665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12863.335342] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12875.079881] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12882.728492] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12891.222916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12899.338123] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [12907.954841] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12950.499156] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12950.504752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12950.509457] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12950.527642] Lustre: Skipped 1 previous similar message [12950.556135] Lustre: Skipped 1 previous similar message [12953.678859] Lustre: server umount lustre-OST0000 complete [12958.689729] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12958.700939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12965.144581] Lustre: server umount lustre-MDT0000 complete [12965.878171] LustreError: 291689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12965.899133] LustreError: 291689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12970.242920] LustreError: 291670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785890188 with bad export cookie 17053025557820730819 [12970.253670] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12970.638378] Lustre: server umount lustre-MDT0001 complete [12988.107702] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [12992.555835] Key type lgssc unregistered [12992.894540] LNet: 295654:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12992.899742] LNetError: 295654:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12992.915853] LNet: Removed LNI 192.168.202.113@tcp [12994.095655] Key type .llcrypt unregistered [12994.099307] Key type ._llcrypt unregistered [13043.673795] Key type ._llcrypt registered [13043.677966] Key type .llcrypt registered [13043.892249] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [13066.501400] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [13067.894038] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13068.023100] alg: No test for adler32 (adler32-zlib) [13069.780140] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [13070.182820] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [13072.064193] Key type lgssc registered [13074.183249] Lustre: Echo OBD driver; http://www.lustre.org/ [13125.313405] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13125.340850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13126.763527] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13126.842442] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13127.005400] Lustre: lustre-MDT0000: new disk, initializing [13127.191197] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13127.260107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13133.687451] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13148.314129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13148.500812] Lustre: 299617:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13148.599633] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13148.603584] Lustre: Skipped 1 previous similar message [13148.698715] Lustre: lustre-MDT0001: new disk, initializing [13148.778215] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13148.823851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13148.834305] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13154.840282] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13168.270470] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13175.791762] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13183.509305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13183.839228] Lustre: lustre-OST0000: new disk, initializing [13183.841523] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13183.861800] Lustre: 301587:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13183.942329] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13191.274230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13191.292102] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13191.426032] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13191.441988] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13204.294779] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13212.081567] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [13227.279226] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [13234.259167] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [13283.822717] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13283.823030] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13283.825602] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13283.843383] Lustre: Skipped 1 previous similar message [13286.473639] Lustre: server umount lustre-OST0000 complete [13292.002047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13292.016972] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13292.053990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13292.067081] Lustre: Skipped 1 previous similar message [13294.058466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13294.063741] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13297.873882] Lustre: server umount lustre-MDT0000 complete [13302.468897] LustreError: 299610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785890520 with bad export cookie 9681348818172839033 [13302.488178] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13303.364600] Lustre: server umount lustre-MDT0001 complete [13313.904631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13320.672837] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13326.696793] Lustre: server umount MGS complete [13340.507630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13340.984893] LustreError: 305503:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13341.004594] LustreError: 305503:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13341.007478] LustreError: 305516:0:(qmt_pool.c:1615:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [13341.087315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13346.275877] LustreError: 305504:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13346.302083] LustreError: 305504:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13346.661131] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13351.398975] LustreError: 305503:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13356.308251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13356.514644] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [13356.528548] Lustre: Skipped 1 previous similar message [13356.715984] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13362.081477] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13370.996230] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13378.446437] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13387.589358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13388.072406] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13395.933530] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13405.482223] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13447.657697] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13447.663534] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13447.665333] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13447.685044] Lustre: Skipped 1 previous similar message [13452.684717] Lustre: server umount lustre-OST0000 complete [13457.894087] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13457.901973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13457.908436] Lustre: Skipped 1 previous similar message [13457.922467] Lustre: Skipped 2 previous similar messages [13472.224435] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13472.641562] Lustre: server umount lustre-MDT0000 complete [13473.252612] LustreError: 305504:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13473.271307] LustreError: 305504:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13477.078846] LustreError: 305484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785890695 with bad export cookie 9681348818172840874 [13477.087786] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13477.102554] LustreError: 305484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13477.643708] Lustre: server umount lustre-MDT0001 complete [13491.861186] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [13494.633472] Key type lgssc unregistered [13494.987512] LNet: 309478:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13494.999861] LNetError: 309478:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13496.038916] LNet: Removed LNI 192.168.202.113@tcp [13496.928350] Key type .llcrypt unregistered [13496.929846] Key type ._llcrypt unregistered [13522.787818] Key type ._llcrypt registered [13522.794676] Key type .llcrypt registered [13522.966428] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [13539.334843] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [13540.416280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13540.484327] alg: No test for adler32 (adler32-zlib) [13541.714390] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [13542.087618] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [13543.784178] Key type lgssc registered [13545.415687] Lustre: Echo OBD driver; http://www.lustre.org/ [13577.443654] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [13595.136418] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13595.189611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13596.555697] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13596.615752] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13596.780154] Lustre: lustre-MDT0000: new disk, initializing [13596.914737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13596.949941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13602.543307] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13614.309301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13614.607376] Lustre: lustre-OST0000: new disk, initializing [13614.612040] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13614.619264] Lustre: Skipped 1 previous similar message [13614.626519] Lustre: 313555:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13614.697217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13622.461705] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [13624.376250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [13624.386151] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [13624.413281] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400