[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 461910177 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002383] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.007015] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.008007] pid_max: default: 32768 minimum: 301 [ 0.009132] LSM: Security Framework initializing [ 0.010033] Yama: becoming mindful. [ 0.011027] SELinux: Initializing. [ 0.012049] *** VALIDATE selinux *** [ 0.019644] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024023] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025120] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026077] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027093] *** VALIDATE tmpfs *** [ 0.028414] *** VALIDATE proc *** [ 0.030148] *** VALIDATE cgroup *** [ 0.031005] *** VALIDATE cgroup2 *** [ 0.032213] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033122] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.035023] Spectre V2 : User space: Vulnerable [ 0.036005] Speculative Store Bypass: Vulnerable [ 0.039789] debug: unmapping init [mem 0xffffffff8f859000-0xffffffff8f860fff] [ 0.041788] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042673] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043026] ... version: 2 [ 0.044009] ... bit width: 48 [ 0.045031] ... generic registers: 4 [ 0.046009] ... value mask: 0000ffffffffffff [ 0.047007] ... max period: 00007fffffffffff [ 0.048019] ... fixed-purpose events: 3 [ 0.048897] ... event mask: 000000070000000f [ 0.049327] rcu: Hierarchical SRCU implementation. [ 0.051701] smp: Bringing up secondary CPUs ... [ 0.052540] x86: Booting SMP configuration: [ 0.053012] .... node #0, CPUs: #1 #2 #3 [ 0.061014] smp: Brought up 1 node, 4 CPUs [ 0.062969] smpboot: Max logical packages: 1 [ 0.063028] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.149188] node 0 deferred pages initialised in 84ms [ 0.152035] devtmpfs: initialized [ 0.153250] x86/mm: Memory block size: 128MB [ 0.155688] gcov: version magic: 0x41383552 [ 0.158249] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.159077] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.160279] pinctrl core: initialized pinctrl subsystem [ 0.161210] [ 0.161761] ************************************************************* [ 0.163013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.165010] ** ** [ 0.168014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.171010] ** ** [ 0.173009] ** This means that this kernel is built to expose internal ** [ 0.174006] ** IOMMU data structures, which may compromise security on ** [ 0.176008] ** your system. ** [ 0.178007] ** ** [ 0.180008] ** If you see this message and you are not debugging the ** [ 0.182008] ** kernel, report this immediately to your vendor! ** [ 0.186011] ** ** [ 0.188017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.190008] ************************************************************* [ 0.192839] NET: Registered protocol family 16 [ 0.195464] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.197063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.200064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.203516] cpuidle: using governor menu [ 0.204622] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.206398] PCI: Using configuration type 1 for base access [ 0.208179] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.218180] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.219070] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.221061] cryptd: max_cpu_qlen set to 1000 [ 0.223190] ACPI: Added _OSI(Module Device) [ 0.225014] ACPI: Added _OSI(Processor Device) [ 0.226010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.227013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.233724] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.240485] ACPI: Interpreter enabled [ 0.242041] ACPI: PM: (supports S0 S3 S4 S5) [ 0.243008] ACPI: Using IOAPIC for interrupt routing [ 0.244103] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.247362] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.257909] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.259023] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.262013] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.264070] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.269375] acpiphp: Slot [2] registered [ 0.270125] acpiphp: Slot [5] registered [ 0.272111] acpiphp: Slot [6] registered [ 0.273111] acpiphp: Slot [7] registered [ 0.274032] acpiphp: Slot [8] registered [ 0.274938] acpiphp: Slot [9] registered [ 0.276066] acpiphp: Slot [10] registered [ 0.277062] acpiphp: Slot [3] registered [ 0.278020] acpiphp: Slot [4] registered [ 0.278959] acpiphp: Slot [11] registered [ 0.280051] acpiphp: Slot [12] registered [ 0.280983] acpiphp: Slot [13] registered [ 0.281063] acpiphp: Slot [14] registered [ 0.281992] acpiphp: Slot [15] registered [ 0.283053] acpiphp: Slot [16] registered [ 0.283991] acpiphp: Slot [17] registered [ 0.285051] acpiphp: Slot [18] registered [ 0.285933] acpiphp: Slot [19] registered [ 0.287058] acpiphp: Slot [20] registered [ 0.288011] acpiphp: Slot [21] registered [ 0.289094] acpiphp: Slot [22] registered [ 0.290063] acpiphp: Slot [23] registered [ 0.290976] acpiphp: Slot [24] registered [ 0.292077] acpiphp: Slot [25] registered [ 0.294097] acpiphp: Slot [26] registered [ 0.295081] acpiphp: Slot [27] registered [ 0.296123] acpiphp: Slot [28] registered [ 0.298088] acpiphp: Slot [29] registered [ 0.299086] acpiphp: Slot [30] registered [ 0.300112] acpiphp: Slot [31] registered [ 0.301086] PCI host bridge to bus 0000:00 [ 0.302015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.304016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.306015] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.307027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.311110] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.314041] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.316145] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.317985] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.321179] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.329014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.333444] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.335011] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.336009] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.338033] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.340727] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.343690] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.344027] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.346665] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.351009] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.361829] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.366010] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.372266] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.377012] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.382012] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.401015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.419764] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.433014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.442017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.464019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.477077] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.481052] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.494018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.507015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.514932] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.520015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.525012] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.541016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.550938] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.566017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.572014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.586015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.591000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.593000] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.593000] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.604020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.615539] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.617293] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.619358] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.623381] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.625198] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.630087] iommu: Default domain type: Passthrough [ 0.631422] SCSI subsystem initialized [ 0.633066] ACPI: bus type USB registered [ 0.643138] usbcore: registered new interface driver usbfs [ 0.645064] usbcore: registered new interface driver hub [ 0.648161] usbcore: registered new device driver usb [ 0.650180] pps_core: LinuxPPS API ver. 1 registered [ 0.655012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.657052] PTP clock support registered [ 0.660037] EDAC MC: Ver: 3.0.0 [ 0.662175] PCI: Using ACPI for IRQ routing [ 0.667777] NetLabel: Initializing [ 0.670015] NetLabel: domain hash size = 128 [ 0.672115] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.676089] NetLabel: unlabeled traffic allowed by default [ 0.678115] vgaarb: loaded [ 0.681471] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.684010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.691654] clocksource: Switched to clocksource kvm-clock [ 0.812022] VFS: Disk quotas dquot_6.6.0 [ 0.813620] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.816492] *** VALIDATE ramfs *** [ 0.817632] *** VALIDATE hugetlbfs *** [ 0.819319] pnp: PnP ACPI init [ 0.821730] pnp: PnP ACPI: found 6 devices [ 0.839440] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.842803] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.844464] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.846607] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.849127] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.851978] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.855471] NET: Registered protocol family 2 [ 0.857894] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.862976] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.867074] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.872863] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.876487] TCP: Hash tables configured (established 65536 bind 65536) [ 0.879814] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.883423] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.886888] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.890734] NET: Registered protocol family 1 [ 0.893403] RPC: Registered named UNIX socket transport module. [ 0.895680] RPC: Registered udp transport module. [ 0.897634] RPC: Registered tcp transport module. [ 0.899421] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.902316] NET: Registered protocol family 44 [ 0.904072] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.906420] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.908651] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.911432] PCI: CLS 0 bytes, default 64 [ 0.913210] Unpacking initramfs... [ 2.564452] debug: unmapping init [mem 0xffff8f8b7cc54000-0xffff8f8b7ffbffff] [ 2.568745] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.570686] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.573319] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.074865] Initialise system trusted keyrings [ 3.076239] Key type blacklist registered [ 3.077825] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.086944] zbud: loaded [ 3.089684] *** VALIDATE nfs *** [ 3.090602] *** VALIDATE nfs4 *** [ 3.091888] pstore: using deflate compression [ 3.095293] Platform Keyring initialized [ 3.206594] NET: Registered protocol family 38 [ 3.208438] Key type asymmetric registered [ 3.210070] Asymmetric key parser 'x509' registered [ 3.211859] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.214833] io scheduler mq-deadline registered [ 3.219789] io scheduler kyber registered [ 3.221083] io scheduler bfq registered [ 3.222879] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.225814] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.228705] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.230839] ACPI: Power Button [PWRF] [ 3.236396] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.243986] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.256457] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.263787] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.280096] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.308846] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.337723] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.341642] Non-volatile memory driver v1.3 [ 3.342976] Linux agpgart interface v0.103 [ 3.373780] virtio_blk virtio1: [vda] 134728 512-byte logical blocks (69.0 MB/65.8 MiB) [ 3.376783] vda: detected capacity change from 0 to 68980736 [ 3.444113] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.448331] vdb: detected capacity change from 0 to 1073741824 [ 3.466385] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.470524] vdc: detected capacity change from 0 to 2621440000 [ 3.485392] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.487534] vdd: detected capacity change from 0 to 2621440000 [ 3.499659] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.502621] vde: detected capacity change from 0 to 4294967296 [ 3.518383] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.522109] vdf: detected capacity change from 0 to 4294967296 [ 3.528985] libphy: Fixed MDIO Bus: probed [ 3.534895] usbcore: registered new interface driver usbserial_generic [ 3.537805] usbserial: USB Serial support registered for generic [ 3.540114] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.544830] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.546936] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.549799] mousedev: PS/2 mouse device common for all mice [ 3.554389] rtc_cmos 00:05: RTC can wake from S4 [ 3.558085] rtc_cmos 00:05: registered as rtc0 [ 3.559579] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.562265] intel_pstate: CPU model not supported [ 3.565041] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.569071] hid: raw HID events driver (C) Jiri Kosina [ 3.573192] usbcore: registered new interface driver usbhid [ 3.575274] usbhid: USB HID core driver [ 3.577674] drop_monitor: Initializing network drop monitor service [ 3.579640] Initializing XFRM netlink socket [ 3.581155] NET: Registered protocol family 10 [ 3.584393] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.591774] Segment Routing with IPv6 [ 3.593055] NET: Registered protocol family 17 [ 3.594808] mpls_gso: MPLS GSO support [ 3.598218] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.604499] RAS: Correctable Errors collector initialized. [ 3.606503] AVX version of gcm_enc/dec engaged. [ 3.608468] AES CTR mode by8 optimization enabled [ 3.707328] sched_clock: Marking stable (3707251327, 0)->(4524848736, -817597409) [ 3.713429] registered taskstats version 1 [ 3.715571] Loading compiled-in X.509 certificates [ 3.722352] zswap: loaded using pool lzo/zbud [ 3.749674] Key type big_key registered [ 3.769383] Key type encrypted registered [ 3.771682] ima: No TPM chip found, activating TPM-bypass! [ 3.773278] ima: Allocated hash algorithm: sha1 [ 3.780116] ima: No architecture policies found [ 3.782617] evm: Initialising EVM extended attributes: [ 3.784950] evm: security.selinux [ 3.786707] evm: security.ima [ 3.787934] evm: security.capability [ 3.790136] evm: HMAC attrs: 0x1 [ 3.793225] rtc_cmos 00:05: setting system clock to 2026-02-18 18:29:03 UTC (1771439343) [ 3.803205] debug: unmapping init [mem 0xffffffff90803000-0xffffffff909fffff] [ 3.808367] debug: unmapping init [mem 0xffffffff8f582000-0xffffffff8f858fff] [ 3.818069] Write protecting the kernel read-only data: 28672k [ 3.822439] debug: unmapping init [mem 0xffffffff8dc03000-0xffffffff8ddfffff] [ 3.825745] debug: unmapping init [mem 0xffffffff8e514000-0xffffffff8e5fffff] [ 3.862425] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.873568] systemd[1]: Detected virtualization kvm. [ 3.875832] systemd[1]: Detected architecture x86-64. [ 3.877972] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.910023] systemd[1]: No hostname configured. [ 3.911940] systemd[1]: Set hostname to . [ 3.914384] random: systemd: uninitialized urandom read (16 bytes read) [ 3.919314] systemd[1]: Initializing machine ID from random generator. [ 4.014722] random: ln: uninitialized urandom read (6 bytes read) [ 4.126568] random: systemd: uninitialized urandom read (16 bytes read) [ 4.129356] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.141607] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.151626] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.090864] device-mapper: uevent: version 1.0.3 [ 5.092961] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 6.611465] virtio_net virtio0 ens2: renamed from eth0 [ 6.667300] random: fast init done [ 6.989704] scsi host0: ata_piix [ 7.022276] scsi host1: ata_piix [ 7.023522] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.031973] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 12.308947] random: crng init done [ 12.310553] random: 7 urandom warning(s) missed due to ratelimiting [ 12.907134] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 14.030878] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.313584] printk: systemd: 26 output lines suppressed due to ratelimiting [ 15.715511] SELinux: Disabled at runtime. [ 15.783358] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.791972] systemd[1]: Detected virtualization kvm. [ 15.794218] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 16.738314] systemd[1]: initrd-switch-root.service: Succeeded. [ 16.743848] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 16.758466] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 16.762835] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 16.767396] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 16.787708] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.808362] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Huge Pages File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Local Encrypted Volumes. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Listening on initctl Compatibility Named Pipe. [ 17.071623] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Mounting Kernel Debug File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 17.644453] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 18.226685] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 18.276863] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 18.450737] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 18.475996] EDAC sbridge: Ver: 1.1.2 [ 20.847668] Key type dns_resolver registered [ 21.238950] NFS: Registering the id_resolver key type [ 21.241317] Key type id_resolver registered [ 21.242583] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg406-server login: [ 46.333386] libcfs: loading out-of-tree module taints kernel. [ 46.348977] Key type ._llcrypt registered [ 46.349989] Key type .llcrypt registered [ 46.385883] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 53.134787] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 53.749034] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 53.755096] alg: No test for adler32 (adler32-zlib) [ 54.750643] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 55.036747] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 56.648208] Key type lgssc registered [ 57.188955] Lustre: Echo OBD driver; http://www.lustre.org/ [ 63.221725] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 77.790877] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 82.398862] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 82.406392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 83.504622] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 83.516070] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 83.559488] Lustre: lustre-MDT0000: new disk, initializing [ 83.583434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 83.589521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 84.855756] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 89.811975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 89.843661] Lustre: 6483:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 89.853281] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 89.854722] Lustre: Skipped 1 previous similar message [ 89.888197] Lustre: lustre-MDT0001: new disk, initializing [ 89.908500] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 89.916215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 89.919691] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 91.116858] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 93.346046] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 96.337431] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 96.415945] Lustre: lustre-OST0000: new disk, initializing [ 96.418202] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 96.432560] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 98.220157] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 100.844370] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 100.846933] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 100.865791] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 103.532753] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 103.575622] Lustre: lustre-OST0001: new disk, initializing [ 103.578077] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 103.595779] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 105.464508] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 106.158029] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 106.160804] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 106.175757] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 111.198857] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 117.045558] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 124.028423] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing check_logdir /tmp/testlogs/ [ 125.706483] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing yml_node [ 127.649778] Lustre: DEBUG MARKER: Client: 2.17.50.152 [ 128.654691] Lustre: DEBUG MARKER: MDS: 2.17.50.152 [ 129.638611] Lustre: DEBUG MARKER: OSS: 2.17.50.152 [ 130.267510] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Feb 18 13:31:09 EST 2026 [ 136.944761] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 137.526759] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 151.520622] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 151.523689] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 151.529099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 152.544817] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 152.548548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 152.552311] Lustre: Skipped 2 previous similar messages [ 152.554116] Lustre: Skipped 2 previous similar messages [ 155.554087] Lustre: server umount lustre-MDT0000 complete [ 157.255380] LustreError: 7414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439496 with bad export cookie 18157552703126341027 [ 157.256778] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 157.261473] LustreError: 7414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 157.664854] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 157.665828] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 157.670420] Lustre: Skipped 1 previous similar message [ 157.672440] Lustre: Skipped 1 previous similar message [ 162.784804] LustreError: 6489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 162.787428] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 162.793406] LustreError: 6489:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 162.795890] Lustre: Skipped 2 previous similar messages [ 163.444140] Lustre: server umount lustre-MDT0001 complete [ 165.468178] Lustre: server umount lustre-OST0000 complete [ 166.889116] Lustre: server umount lustre-OST0001 complete [ 169.360573] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 172.795537] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 186.939182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 187.059285] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 187.078926] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 187.122530] Lustre: lustre-MDT0000: new disk, initializing [ 187.160926] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 187.170637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 188.779519] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 193.775676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.834224] Lustre: 14948:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 194.000832] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 195.693905] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 199.571501] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 201.032988] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 204.111278] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 204.232741] Lustre: lustre-OST0000: new disk, initializing [ 204.235106] Lustre: Skipped 1 previous similar message [ 204.237742] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 204.240695] Lustre: Skipped 2 previous similar messages [ 204.277155] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 204.280619] Lustre: Skipped 1 previous similar message [ 205.835915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 205.847114] Lustre: Skipped 1 previous similar message [ 205.852278] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 205.902615] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 206.540792] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 210.017888] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 211.693939] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 211.775355] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 213.648635] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 213.735584] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 216.035963] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 216.039729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 216.041295] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 216.044636] LustreError: Skipped 1 previous similar message [ 216.049294] Lustre: Skipped 1 previous similar message [ 220.566987] Lustre: server umount lustre-OST0000 complete [ 224.736915] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 224.743124] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 224.748144] Lustre: Skipped 1 previous similar message [ 224.750352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 228.393111] Lustre: server umount lustre-MDT0000 complete [ 229.960736] LustreError: 16688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439569 with bad export cookie 18157552703126342546 [ 229.964318] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 229.969037] LustreError: 16688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 230.093276] Lustre: server umount lustre-MDT0001 complete [ 232.156111] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 13:32:51 (1771439571) [ 234.880234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 235.086913] LustreError: 18802:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 235.121688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 236.664802] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 240.376189] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 242.204898] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 244.320906] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 245.721830] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 248.256206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 250.506622] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 252.518720] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 265.144804] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 270.306777] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 270.307877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 270.312352] Lustre: Skipped 2 previous similar messages [ 270.314671] Lustre: Skipped 2 previous similar messages [ 274.622182] Lustre: server umount lustre-MDT0000 complete [ 275.431195] LustreError: 19512:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 275.440021] LustreError: 19512:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 276.336812] LustreError: 18781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439616 with bad export cookie 18157552703126343428 [ 276.344506] LustreError: 18781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 276.344655] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 276.505219] Lustre: server umount lustre-MDT0001 complete [ 288.687446] Lustre: server umount lustre-OST0000 complete [ 297.449689] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 302.103052] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 302.299912] LustreError: 22851:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 302.335508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 302.338274] Lustre: Skipped 2 previous similar messages [ 304.142065] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 307.685402] LustreError: 22851:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 308.202728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 310.210645] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 311.627211] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 314.788344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 317.882362] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 321.733524] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 321.812275] Lustre: lustre-OST0001: new disk, initializing [ 321.815671] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 323.789072] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 323.793830] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 323.851103] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 324.250404] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 329.609748] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 331.243960] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 343.521291] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 343.522967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 343.526603] Lustre: Skipped 4 previous similar messages [ 343.530939] Lustre: Skipped 5 previous similar messages [ 347.743719] Lustre: server umount lustre-MDT0000 complete [ 348.642261] LustreError: 22851:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 348.648516] LustreError: 22851:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 349.085259] LustreError: 25115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439688 with bad export cookie 18157552703126344275 [ 349.088906] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 349.089056] LustreError: 25115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 349.205802] Lustre: server umount lustre-MDT0001 complete [ 360.971839] Lustre: server umount lustre-OST0000 complete [ 372.715541] Lustre: server umount lustre-OST0001 complete [ 378.634640] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 382.589910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 382.775570] LustreError: 28100:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 382.814411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 382.818025] Lustre: Skipped 3 previous similar messages [ 384.294786] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 387.623771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 389.240266] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 390.391929] Lustre: 29215:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 393.219541] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 395.761537] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 399.613281] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 402.166377] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 411.136625] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 412.763383] Lustre: 31063:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 422.882394] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 422.885226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 422.887289] Lustre: Skipped 3 previous similar messages [ 422.890945] Lustre: Skipped 3 previous similar messages [ 425.883949] Lustre: server umount lustre-MDT0000 complete [ 427.625895] LustreError: 28079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439767 with bad export cookie 18157552703126345570 [ 427.632851] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 427.635684] LustreError: 28079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 427.795186] Lustre: server umount lustre-MDT0001 complete [ 439.759331] Lustre: server umount lustre-OST0000 complete [ 451.800686] Lustre: server umount lustre-OST0001 complete [ 454.932112] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 13:36:33 (1771439793) [ 465.085769] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 468.603067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 468.738051] LustreError: 33844:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 468.745563] LustreError: 33844:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 470.007439] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 472.995694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 474.507047] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 475.537027] Lustre: 34956:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 477.838393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 479.981463] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 483.127178] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 485.276773] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 488.666420] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 490.041247] Lustre: 36806:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 498.659123] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 498.659359] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 498.664930] Lustre: Skipped 1 previous similar message [ 502.493398] Lustre: server umount lustre-MDT0000 complete [ 503.829723] LustreError: 33824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439843 with bad export cookie 18157552703126346676 [ 503.832635] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 503.833600] LustreError: 33824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 503.965655] Lustre: server umount lustre-MDT0001 complete [ 516.115220] Lustre: server umount lustre-OST0000 complete [ 527.838498] Lustre: server umount lustre-OST0001 complete [ 531.021374] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 13:37:49 (1771439869) [ 536.972176] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 540.801247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 540.962174] LustreError: 38929:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 540.971671] LustreError: 38929:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 540.985635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 540.988733] Lustre: Skipped 7 previous similar messages [ 542.315436] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 545.310404] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 546.870785] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 548.004966] Lustre: 40042:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 550.451851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 552.779571] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 556.137542] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 558.711650] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 562.391133] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 569.221413] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 569.224082] Lustre: Skipped 1 previous similar message [ 570.330253] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.6@tcp (stopping) [ 570.332258] Lustre: Skipped 4 previous similar messages [ 571.361817] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 576.488401] Lustre: server umount lustre-MDT0000 complete [ 577.930573] LustreError: 38909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439917 with bad export cookie 18157552703126347782 [ 577.934437] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 577.935360] LustreError: 38909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 592.352166] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 592.462777] Lustre: server umount lustre-MDT0001 complete [ 595.664539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 602.592684] LustreError: 3635:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8f8bd0686d80 x1857488834313088/t0(0) o250->MGC192.168.204.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 604.110779] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 607.104914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 608.677821] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 610.698338] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 618.475498] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 622.714468] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 11 sec [ 624.027203] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 624.696716] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 627.681831] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 627.685280] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 627.689822] Lustre: Skipped 6 previous similar messages [ 632.286885] Lustre: server umount lustre-MDT0000 complete [ 633.582381] LustreError: 38909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771439973 with bad export cookie 18157552703126348958 [ 633.586938] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 633.587506] LustreError: 38909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 633.712828] Lustre: server umount lustre-MDT0001 complete [ 645.193479] Lustre: server umount lustre-OST0000 complete [ 656.740613] Lustre: server umount lustre-OST0001 complete [ 659.885624] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 13:39:58 (1771439998) [ 664.964602] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 668.575759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 670.053441] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 673.041838] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 674.497727] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 675.652728] Lustre: 47399:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 675.657067] Lustre: 47399:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 678.067206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 679.269310] LustreError: 47935:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 679.276140] LustreError: 47935:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 18 previous similar messages [ 680.318607] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 683.366307] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 685.562449] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 688.746754] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 695.360297] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 703.969970] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 706.140987] Lustre: server umount lustre-MDT0000 complete [ 707.319289] LustreError: 47018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440047 with bad export cookie 18157552703126349749 [ 707.321510] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 707.323822] LustreError: 47018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 707.447382] Lustre: server umount lustre-MDT0001 complete [ 719.303149] Lustre: server umount lustre-OST0000 complete [ 730.904638] Lustre: server umount lustre-OST0001 complete [ 734.100172] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 13:41:13 (1771440073) [ 734.942520] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 735.581729] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 13:41:14 (1771440074) [ 739.053670] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 742.001973] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 749.869049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 749.966053] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 750.001794] Lustre: lustre-MDT0000: new disk, initializing [ 750.032886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 751.467675] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 754.815085] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 757.206882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 759.116165] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 759.132929] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 759.358920] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 762.542666] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 763.286922] Lustre: DEBUG MARKER: create test files [ 774.625320] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 780.420702] Lustre: server umount lustre-OST0000 complete [ 781.805499] Lustre: server umount lustre-MDT0000 complete [ 785.522384] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 786.645585] Key type lgssc unregistered [ 786.788149] LNet: 54788:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 786.791308] LNetError: 54788:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 786.803482] LNet: Removed LNI 192.168.204.106@tcp [ 787.143142] Key type .llcrypt unregistered [ 787.144683] Key type ._llcrypt unregistered [ 797.943598] Key type ._llcrypt registered [ 797.944672] Key type .llcrypt registered [ 797.984776] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 798.333694] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 798.408381] alg: No test for adler32 (adler32-zlib) [ 799.280017] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 799.371295] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 800.960139] Key type lgssc registered [ 801.347836] Lustre: Echo OBD driver; http://www.lustre.org/ [ 803.922868] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 803.931340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 805.094348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 806.311873] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 808.019642] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 810.291892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 810.416697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 812.277916] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 814.029287] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 814.500561] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 814.522971] Lustre: 57367:0:(osd_internal.h:1455:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 814.525146] Lustre: 57367:0:(osd_handler.c:2093:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 814.526910] Lustre: 57367:0:(osd_handler.c:2100:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 814.528729] Lustre: 57367:0:(osd_handler.c:2110:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 814.530655] Lustre: 57367:0:(osd_handler.c:2117:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 814.532778] Lustre: 57367:0:(osd_handler.c:2124:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 822.482664] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 824.049407] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 829.921661] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 829.922057] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 829.929786] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 832.017306] Lustre: server umount lustre-OST0000 complete [ 833.383646] Lustre: server umount lustre-MDT0000 complete [ 836.666340] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 837.657691] Key type lgssc unregistered [ 837.786399] LNet: 58847:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 837.788955] LNetError: 58847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 837.797629] LNet: Removed LNI 192.168.204.106@tcp [ 838.078119] Key type .llcrypt unregistered [ 838.079874] Key type ._llcrypt unregistered [ 849.153841] Key type ._llcrypt registered [ 849.155277] Key type .llcrypt registered [ 849.191468] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 849.585871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 849.594619] alg: No test for adler32 (adler32-zlib) [ 850.451072] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 850.539190] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 852.112141] Key type lgssc registered [ 852.448358] Lustre: Echo OBD driver; http://www.lustre.org/ [ 854.927692] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 854.937048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 856.081628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 857.374741] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 859.186900] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 861.378570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 861.484923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 863.267847] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 864.983260] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 865.573491] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 865.594674] Lustre: 62078:0:(osd_internal.h:1455:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 865.598794] Lustre: 62078:0:(osd_handler.c:2093:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 865.601835] Lustre: 62078:0:(osd_handler.c:2100:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 865.604238] Lustre: 62078:0:(osd_handler.c:2110:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 865.606884] Lustre: 62078:0:(osd_handler.c:2117:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 865.609941] Lustre: 62078:0:(osd_handler.c:2124:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 873.188681] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 876.001403] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 876.001505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 876.010275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 877.582513] Lustre: server umount lustre-OST0000 complete [ 884.981778] Lustre: server umount lustre-MDT0000 complete [ 888.529403] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 889.486203] Key type lgssc unregistered [ 889.611282] LNet: 62951:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 889.613621] LNetError: 62951:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 889.621283] LNet: Removed LNI 192.168.204.106@tcp [ 889.896764] Key type .llcrypt unregistered [ 889.897790] Key type ._llcrypt unregistered [ 895.627838] Key type ._llcrypt registered [ 895.628753] Key type .llcrypt registered [ 895.669721] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 901.869914] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 902.249399] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 902.284588] alg: No test for adler32 (adler32-zlib) [ 903.149865] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 903.239349] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 904.824208] Key type lgssc registered [ 905.213407] Lustre: Echo OBD driver; http://www.lustre.org/ [ 916.700412] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 916.706154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 917.810606] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 917.822752] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 917.861528] Lustre: lustre-MDT0000: new disk, initializing [ 917.887773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 917.897160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 919.193957] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 923.192693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 923.235129] Lustre: 66046:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 923.258412] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 923.260245] Lustre: Skipped 1 previous similar message [ 923.300546] Lustre: lustre-MDT0001: new disk, initializing [ 923.335700] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 923.348390] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 923.351271] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 924.598708] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 927.588469] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 928.722538] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 930.958822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 931.071790] Lustre: lustre-OST0000: new disk, initializing [ 931.073744] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 931.101611] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 932.869077] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 933.050127] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 933.054632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 933.097112] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 935.937348] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 937.246823] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 937.326202] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 938.626365] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 938.698154] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 943.586234] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 943.588048] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 943.589427] LustreError: Skipped 1 previous similar message [ 943.593571] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 943.596646] Lustre: Skipped 1 previous similar message [ 945.418702] Lustre: server umount lustre-OST0000 complete [ 948.705602] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 948.706243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 948.709342] Lustre: Skipped 2 previous similar messages [ 952.727851] Lustre: server umount lustre-MDT0000 complete [ 953.947423] LustreError: 66038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440293 with bad export cookie 4842057589806140754 [ 953.949818] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 953.951663] LustreError: 66038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 954.062355] Lustre: server umount lustre-MDT0001 complete [ 956.900579] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 13:44:55 (1771440295) [ 960.169249] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 972.943530] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 975.905458] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 987.544883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 987.646719] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 987.658882] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 987.697579] Lustre: lustre-MDT0000: new disk, initializing [ 987.723309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 987.730607] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 989.015542] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 993.196142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 993.234928] Lustre: 73596:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 993.265479] LustreError: 73607:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 993.267378] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 993.270660] LustreError: 73607:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 993.272979] Lustre: Skipped 1 previous similar message [ 993.310822] Lustre: lustre-MDT0001: new disk, initializing [ 993.344508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 993.358736] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 993.362560] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 994.667480] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 997.845301] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 999.121933] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1001.391516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1001.500812] Lustre: lustre-OST0000: new disk, initializing [ 1001.502716] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1001.527172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1002.871260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1002.875557] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1002.914293] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1003.385760] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1006.439172] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1007.766243] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1007.840399] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1009.097073] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1009.167409] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1013.217845] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1013.219345] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1013.220404] LustreError: Skipped 1 previous similar message [ 1013.224124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1013.227081] Lustre: Skipped 2 previous similar messages [ 1015.884899] Lustre: server umount lustre-OST0000 complete [ 1018.337795] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1018.338255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1018.341961] Lustre: Skipped 2 previous similar messages [ 1018.345108] Lustre: Skipped 1 previous similar message [ 1023.197833] Lustre: server umount lustre-MDT0000 complete [ 1023.457214] LustreError: 74908:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1023.463054] LustreError: 74908:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1024.480350] LustreError: 73586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440364 with bad export cookie 4842057589806141622 [ 1024.483598] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1024.485202] LustreError: 73586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1024.609605] Lustre: server umount lustre-MDT0001 complete [ 1027.231135] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 13:46:06 (1771440366) [ 1029.305865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1029.465795] LustreError: 77598:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1029.486342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1030.688013] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1033.328808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1034.662633] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1036.281088] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1037.350455] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1039.471544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1041.448356] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1043.215459] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1044.414762] Lustre: 79931:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting a9c18373-7f25-4674-a4ef-d817d22747c5 at adminstrative request [ 1049.058304] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1049.063172] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1059.300677] Lustre: MGS: Client a9c18373-7f25-4674-a4ef-d817d22747c5 (at 0@lo) reconnecting [ 1059.302848] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x433271b93e61140c to 0x433271b93e61169e [ 1059.307801] Lustre: MGC192.168.204.106@tcp: Connection restored to 0@lo (at 0@lo) [ 1076.702902] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1076.774885] Lustre: lustre-OST0001: new disk, initializing [ 1076.776602] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1076.804780] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1076.806843] Lustre: Skipped 2 previous similar messages [ 1078.511883] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1081.443414] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1082.450887] Lustre: server umount lustre-OST0001 complete [ 1082.880831] LustreError: 79189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1084.897356] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1084.900507] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1084.905788] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1086.944821] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1087.971935] LustreError: 79189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1087.979669] LustreError: 79189:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1093.093432] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1093.095372] Lustre: Skipped 4 previous similar messages [ 1098.208121] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1098.214614] LustreError: 79192:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1098.220204] LustreError: 79192:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1098.267755] Lustre: server umount lustre-OST0000 complete [ 1100.256791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1100.259729] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1100.264099] Lustre: Skipped 1 previous similar message [ 1105.555868] Lustre: server umount lustre-MDT0000 complete [ 1106.741820] LustreError: 80327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440446 with bad export cookie 4842057589806143134 [ 1106.746666] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1106.869170] Lustre: server umount lustre-MDT0001 complete [ 1109.521493] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 13:47:28 (1771440448) [ 1110.013530] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1110.562256] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 13:47:29 (1771440449) [ 1111.078307] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1111.603670] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 13:47:30 (1771440450) [ 1112.087042] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1112.607527] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 13:47:31 (1771440451) [ 1113.162063] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1113.714311] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 13:47:32 (1771440452) [ 1114.222382] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1116.159771] loop: module loaded [ 1116.163690] loop0: detected capacity change from 0 to 4294967296 [ 1116.181201] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1116.670059] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1117.586176] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1118.967553] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1127.392273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1127.405771] LustreError: 84311:0:(obd_config.c:880:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1127.408237] LustreError: 84311:0:(obd_mount.c:217:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1127.410630] LustreError: 84311:0:(tgt_mount.c:2547:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1127.413371] LustreError: 84311:0:(super25.c:190:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1133.207232] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1136.179566] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1147.917612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1148.013471] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1148.023627] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1148.059474] Lustre: lustre-MDT0000: new disk, initializing [ 1148.081515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1148.087667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1149.352862] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1153.431048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1153.469878] Lustre: 87137:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1153.564566] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1154.796229] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1157.773681] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1158.866125] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1160.952305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1162.832787] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1164.851789] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1164.881957] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1165.777902] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1167.022516] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1167.087982] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1168.334665] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1168.402706] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1169.890051] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1169.891475] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1169.892687] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1169.895763] Lustre: Skipped 2 previous similar messages [ 1169.897298] Lustre: Skipped 5 previous similar messages [ 1171.087514] Lustre: server umount lustre-OST0000 complete [ 1178.402558] Lustre: server umount lustre-MDT0000 complete [ 1180.130671] LustreError: 87148:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1180.137851] LustreError: 87148:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1180.540655] LustreError: 88879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440520 with bad export cookie 4842057589806143736 [ 1180.542865] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1180.547520] LustreError: 88879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1180.780754] Lustre: server umount lustre-MDT0001 complete [ 1186.793833] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 13:48:44 (1771440524) [ 1191.075354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1194.250599] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1199.892305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1202.872279] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1206.429479] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1208.959227] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1212.993218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1217.139995] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1220.593795] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1223.080739] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1223.255781] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1227.741154] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1227.985726] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1235.402114] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1235.635524] Lustre: lustre-OST0001: new disk, initializing [ 1235.638566] Lustre: Skipped 2 previous similar messages [ 1235.645467] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1235.654629] Lustre: Skipped 3 previous similar messages [ 1240.905843] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1241.705940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1241.709729] Lustre: Skipped 2 previous similar messages [ 1241.712376] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1241.817996] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1246.042751] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1248.622173] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1248.751662] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1251.407798] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1251.530904] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1263.804720] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1276.357949] Lustre: Failing over lustre-MDT0000 [ 1276.725517] Lustre: server umount lustre-MDT0000 complete [ 1276.898557] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1276.905511] LustreError: Skipped 2 previous similar messages [ 1276.908202] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1276.915596] Lustre: Skipped 2 previous similar messages [ 1276.921666] LustreError: 92307:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1276.927519] LustreError: 92307:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1294.114310] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1294.116271] LDISKFS-fs (dm-0): recovery complete [ 1294.120113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1294.224769] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1294.302888] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.6@tcp (not set up) [ 1294.306175] Lustre: Skipped 5 previous similar messages [ 1294.372791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1294.375016] Lustre: Skipped 6 previous similar messages [ 1294.402094] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1296.531149] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1299.411689] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1299.416396] Lustre: 92034:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1299.431449] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1299.474133] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1299.801021] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1300.130886] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1300.138772] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 8 previous similar messages [ 1300.146740] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1300.160674] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1300.480160] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1300.488499] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1301.444491] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1301.453685] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 3 previous similar messages [ 1301.457279] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1301.461535] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1301.776131] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1301.779896] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1303.666938] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1303.673808] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1303.679408] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1303.684253] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1304.002382] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1304.005981] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1307.780526] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1307.784367] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 1307.793079] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1307.797206] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1308.112535] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1308.116114] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1315.978850] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1315.987199] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 1315.995735] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1316.000679] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1316.320102] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1316.322687] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1332.282863] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1332.287579] Lustre: 96771:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 51 previous similar messages [ 1332.290855] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1332.295226] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1332.608277] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1332.612073] LustreError: 96771:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1359.208061] Lustre: 96771:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1359.211025] Lustre: 96771:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1359.213870] Lustre: 96771:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b47b2f85-51dc-4bf6-969b-73d09ba735bc@192.168.204.6@tcp [ 1359.217695] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1359.219648] Lustre: 96771:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000fe64a1bd x1857489716976384/t0(8589934787) o101->b47b2f85-51dc-4bf6-969b-73d09ba735bc@192.168.204.6@tcp:64/0 lens 592/0 e 0 to 0 dl 1771440709 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1359.228339] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1359.232551] Lustre: lustre-MDT0000: Denying connection for new client b47b2f85-51dc-4bf6-969b-73d09ba735bc (at 192.168.204.6@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1359.235563] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1359.254619] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1359.256460] Lustre: Skipped 2 previous similar messages [ 1359.259376] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1359.275560] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1359.275576] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:50 to 0x2c0000401:545) [ 1370.085369] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1370.090529] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1370.094673] Lustre: Skipped 3 previous similar messages [ 1370.098550] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1382.369588] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1382.454168] Lustre: server umount lustre-OST0000 complete [ 1386.467350] LustreError: 94146:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1386.479550] LustreError: 94146:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 23 previous similar messages [ 1390.706778] Lustre: server umount lustre-OST0001 complete [ 1398.911732] Lustre: server umount lustre-MDT0000 complete [ 1401.017694] LustreError: 91122:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440740 with bad export cookie 4842057589806194934 [ 1401.021539] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1401.023982] LustreError: 91122:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1401.237407] Lustre: server umount lustre-MDT0001 complete [ 1406.389811] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 13:52:25 (1771440745) [ 1410.402812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1410.407231] Lustre: *** cfs_fail_loc=197, val=0*** [ 1425.952714] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1431.072476] LustreError: 98732:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.106@tcp: failed processing log, type 4: rc = -110 [ 1436.203565] Lustre: *** cfs_fail_loc=197, val=0*** [ 1436.208521] Lustre: Skipped 65 previous similar messages [ 1459.240715] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1462.528675] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1463.752964] Lustre: server umount lustre-OST0000 complete [ 1468.163203] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 13:53:26 (1771440806) [ 1483.823799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1485.924720] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1489.605299] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1492.031087] LustreError: 100268:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1492.035326] LustreError: 100268:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1492.038294] LustreError: 100268:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1492.193308] Lustre: server umount lustre-MDT0000 complete [ 1505.510978] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1509.887635] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1525.656189] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1525.822042] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1525.836210] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1525.873830] Lustre: lustre-MDT0000: new disk, initializing [ 1525.909300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1527.671697] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1532.811912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1532.861885] Lustre: 104299:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1533.017921] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1534.699311] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1538.353039] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1539.855183] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1543.493519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1546.025585] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1550.004342] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1551.795096] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1553.464717] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1553.931663] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1555.679487] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1555.777091] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1558.504414] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1558.504768] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1558.513555] LustreError: Skipped 3 previous similar messages [ 1558.521434] Lustre: Skipped 5 previous similar messages [ 1558.531165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1558.534497] Lustre: Skipped 14 previous similar messages [ 1562.811669] Lustre: server umount lustre-OST0000 complete [ 1570.920987] Lustre: server umount lustre-MDT0000 complete [ 1573.092866] LustreError: 106038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440912 with bad export cookie 4842057589806205700 [ 1573.099057] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1573.104724] LustreError: 106038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1573.280970] Lustre: server umount lustre-MDT0001 complete [ 1577.638591] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 13:55:16 (1771440916) [ 1582.122745] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 1583.778300] Key type lgssc unregistered [ 1583.952779] LNet: 108371:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1583.958893] LNetError: 108371:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1583.966927] LNet: Removed LNI 192.168.204.106@tcp [ 1584.434512] Key type .llcrypt unregistered [ 1584.436345] Key type ._llcrypt unregistered [ 1592.425327] Key type ._llcrypt registered [ 1592.426957] Key type .llcrypt registered [ 1592.480951] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1600.380947] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1600.918739] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1600.973804] alg: No test for adler32 (adler32-zlib) [ 1601.877565] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 1601.986522] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 1603.592147] Key type lgssc registered [ 1604.102584] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1622.668402] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1622.680454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1623.803179] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1623.823640] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1623.865363] Lustre: lustre-MDT0000: new disk, initializing [ 1623.907890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1623.921113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1625.736668] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1630.997236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1631.124117] Lustre: lustre-OST0000: new disk, initializing [ 1631.126653] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1631.131340] Lustre: Skipped 1 previous similar message [ 1631.159263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1633.687339] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1636.348500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1636.355628] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1636.368414] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1639.087562] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1639.156830] Lustre: lustre-OST0001: new disk, initializing [ 1639.160233] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1639.199151] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1639.584729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1639.590173] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1639.605556] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1641.701395] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1650.146351] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1650.147140] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1650.151785] Lustre: Skipped 1 previous similar message [ 1652.560992] Lustre: server umount lustre-MDT0000 complete [ 1655.003277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1656.755118] LustreError: 112467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771440996 with bad export cookie 2291875533975508952 [ 1656.756887] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1656.759280] LustreError: 112467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1656.800324] Lustre: server umount lustre-OST0000 complete [ 1658.366114] Lustre: server umount lustre-OST0001 complete [ 1661.591365] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 13:56:40 (1771441000) [ 1665.492239] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1674.670926] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 13:56:53 (1771441013) [ 1691.422928] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1695.421200] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1711.556548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1711.671613] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1711.686982] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1711.726304] Lustre: lustre-MDT0000: new disk, initializing [ 1711.770450] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1711.781766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1713.593929] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1719.032756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1719.086429] Lustre: 119988:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1719.122544] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1719.124975] Lustre: Skipped 1 previous similar message [ 1719.169859] Lustre: lustre-MDT0001: new disk, initializing [ 1719.231453] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1719.252441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1719.257554] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1720.908417] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1724.844539] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1726.292744] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1729.181091] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1729.300674] Lustre: lustre-OST0000: new disk, initializing [ 1729.305820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1729.351610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1731.912665] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1733.417891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1733.424892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1733.461890] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1735.709719] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1737.458864] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1737.557703] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1739.425006] hrtimer: interrupt took 1933288 ns [ 1739.530133] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1739.642215] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1743.842862] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1743.843211] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1743.845823] LustreError: Skipped 1 previous similar message [ 1743.846416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1743.850474] Lustre: Skipped 1 previous similar message [ 1743.857654] Lustre: Skipped 2 previous similar messages [ 1746.665773] Lustre: server umount lustre-OST0000 complete [ 1748.963840] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1748.967555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1748.967976] Lustre: Skipped 1 previous similar message [ 1754.082314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1754.087832] Lustre: Skipped 2 previous similar messages [ 1754.404982] Lustre: server umount lustre-MDT0000 complete [ 1755.900821] LustreError: 119980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441095 with bad export cookie 2291875533975510163 [ 1755.903660] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1755.904525] LustreError: 119980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1756.038846] Lustre: server umount lustre-MDT0001 complete [ 1759.377436] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 13:58:18 (1771441098) [ 1762.002111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1762.253278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1763.848888] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1767.393061] LustreError: 123989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1767.399037] LustreError: 123989:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1767.633405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1769.425150] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1771.624147] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1773.164786] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1776.011872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1778.500322] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1781.248206] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1789.183731] Lustre: *** cfs_fail_loc=159, val=0*** [ 1804.774791] Lustre: lustre-MDT0000: Client 1f75f28a-349f-4c07-9392-7cb1ec61df27 (at 192.168.204.6@tcp) reconnecting [ 1805.441699] Lustre: *** cfs_fail_loc=159, val=0*** [ 1805.445025] Lustre: Skipped 5 previous similar messages [ 1820.643290] Lustre: lustre-MDT0000: Client 1f75f28a-349f-4c07-9392-7cb1ec61df27 (at 192.168.204.6@tcp) reconnecting [ 1837.539017] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1837.540352] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1837.545109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1837.550569] Lustre: Skipped 1 previous similar message [ 1842.909440] Lustre: server umount lustre-OST0000 complete [ 1844.704881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1844.707905] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1844.712860] Lustre: Skipped 1 previous similar message [ 1847.778733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1847.781429] Lustre: Skipped 4 previous similar messages [ 1850.611332] Lustre: server umount lustre-MDT0000 complete [ 1852.340345] LustreError: 126316:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441192 with bad export cookie 2291875533975511045 [ 1852.343427] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1852.344858] LustreError: 126316:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1852.468919] Lustre: server umount lustre-MDT0001 complete [ 1856.812534] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 1858.038302] Key type lgssc unregistered [ 1858.175782] LNet: 127413:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1858.179279] LNetError: 127413:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1858.189534] LNet: Removed LNI 192.168.204.106@tcp [ 1858.559607] Key type .llcrypt unregistered [ 1858.561616] Key type ._llcrypt unregistered [ 1869.539174] Key type ._llcrypt registered [ 1869.540613] Key type .llcrypt registered [ 1869.586204] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1869.950348] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1869.981984] alg: No test for adler32 (adler32-zlib) [ 1870.870654] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 1870.990865] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 1872.584160] Key type lgssc registered [ 1873.090863] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1876.350100] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1876.358630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1877.585296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1879.213330] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1882.726498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1882.873322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1884.482970] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1886.647147] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1888.481671] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1891.610598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1891.765246] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1894.276587] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1896.620713] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1903.019866] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1907.960175] Lustre: *** cfs_fail_loc=159, val=0*** [ 1923.559076] Lustre: lustre-MDT0000: Client c14d012b-a832-4bdb-8bcf-af33d5a20f7a (at 192.168.204.6@tcp) reconnecting [ 1924.480847] Lustre: *** cfs_fail_loc=159, val=0*** [ 1939.940639] Lustre: lustre-MDT0001: Client c14d012b-a832-4bdb-8bcf-af33d5a20f7a (at 192.168.204.6@tcp) reconnecting [ 1940.603199] Lustre: *** cfs_fail_loc=159, val=0*** [ 1940.605780] Lustre: Skipped 3 previous similar messages [ 1956.834683] Lustre: lustre-MDT0001: Client c14d012b-a832-4bdb-8bcf-af33d5a20f7a (at 192.168.204.6@tcp) reconnecting [ 1958.386420] Lustre: 131497:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1959.423071] Lustre: *** cfs_fail_loc=159, val=0*** [ 1959.424834] Lustre: Skipped 4 previous similar messages [ 1974.756570] Lustre: lustre-MDT0000: Client ae626625-5d6b-4d1e-9930-3031d829ba7f (at 192.168.204.6@tcp) reconnecting [ 1975.410031] Lustre: *** cfs_fail_loc=159, val=0*** [ 1975.418536] Lustre: Skipped 14 previous similar messages [ 1990.622936] Lustre: lustre-MDT0000: Client ae626625-5d6b-4d1e-9930-3031d829ba7f (at 192.168.204.6@tcp) reconnecting [ 1991.227739] Lustre: 131791:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1995.235568] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1995.236346] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1995.241524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1995.246363] Lustre: Skipped 1 previous similar message [ 1998.814792] Lustre: server umount lustre-OST0000 complete [ 2000.353577] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2000.359484] Lustre: Skipped 1 previous similar message [ 2000.361484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2000.363697] Lustre: Skipped 1 previous similar message [ 2005.473564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2005.476459] Lustre: Skipped 1 previous similar message [ 2006.506900] Lustre: server umount lustre-MDT0000 complete [ 2008.038456] LustreError: 128768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441347 with bad export cookie 2620551384712458772 [ 2008.039496] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2008.043265] LustreError: 128768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2008.174075] Lustre: server umount lustre-MDT0001 complete [ 2012.178679] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2013.261277] Key type lgssc unregistered [ 2013.400324] LNet: 132824:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2013.402905] LNetError: 132824:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2013.411365] LNet: Removed LNI 192.168.204.106@tcp [ 2013.734302] Key type .llcrypt unregistered [ 2013.735707] Key type ._llcrypt unregistered [ 2023.974790] Key type ._llcrypt registered [ 2023.977665] Key type .llcrypt registered [ 2024.025055] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2024.502546] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2024.519709] alg: No test for adler32 (adler32-zlib) [ 2025.383958] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2025.478580] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2027.072222] Key type lgssc registered [ 2027.525563] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2030.434760] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2030.443280] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2031.657822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2033.288040] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2036.892639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2037.056139] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2038.690278] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2040.923820] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2042.346808] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2045.037781] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2045.181217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2047.189361] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2047.277030] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2049.140252] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2056.489307] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2061.986176] Lustre: 136468:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2062.693262] Lustre: 136516:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2063.256906] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2064.511599] Lustre: 136663:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2064.841360] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2064.845018] Lustre: Skipped 1 previous similar message [ 2065.181708] Lustre: 136761:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2066.915371] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2066.915769] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2066.924341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2066.929129] Lustre: Skipped 1 previous similar message [ 2068.369897] Lustre: server umount lustre-OST0000 complete [ 2071.912342] Lustre: server umount lustre-MDT0000 complete [ 2072.033593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2072.034626] LustreError: 134196:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2072.038322] Lustre: Skipped 2 previous similar messages [ 2072.044135] LustreError: 134196:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2073.471361] LustreError: 134179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441413 with bad export cookie 11212140750769019108 [ 2073.473498] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2073.475267] LustreError: 134179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2073.480159] LustreError: 134191:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 2073.611902] Lustre: server umount lustre-MDT0001 complete [ 2077.479737] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2078.591233] Key type lgssc unregistered [ 2078.742581] LNet: 137791:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2078.745859] LNetError: 137791:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2078.756513] LNet: Removed LNI 192.168.204.106@tcp [ 2079.134995] Key type .llcrypt unregistered [ 2079.136972] Key type ._llcrypt unregistered [ 2089.184368] Key type ._llcrypt registered [ 2089.185806] Key type .llcrypt registered [ 2089.236742] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2089.567847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2089.605085] alg: No test for adler32 (adler32-zlib) [ 2090.469449] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2090.546422] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2092.128197] Key type lgssc registered [ 2092.506480] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2095.413699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2095.421339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2096.582181] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2097.887024] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2100.997111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2101.134080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2102.507763] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2104.530421] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2105.721596] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2108.163406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2108.313478] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2110.379202] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2110.552306] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2112.602406] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2122.665386] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2125.484198] Lustre: *** cfs_fail_loc=159, val=0*** [ 2125.485779] Lustre: Skipped 1 previous similar message [ 2140.641621] Lustre: lustre-MDT0000: Client bb2b8399-8b6d-4e0d-8746-bec89b85de5b (at 192.168.204.6@tcp) reconnecting [ 2143.203076] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2143.203278] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2143.203783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2143.206573] Lustre: Skipped 1 previous similar message [ 2148.183411] Lustre: server umount lustre-OST0000 complete [ 2152.418168] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2152.421296] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2152.427094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2152.429718] Lustre: Skipped 1 previous similar message [ 2153.441148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2153.441779] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2155.805484] Lustre: server umount lustre-MDT0000 complete [ 2157.174150] LustreError: 140789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441496 with bad export cookie 6098903674141599020 [ 2157.177084] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2157.178625] LustreError: 140789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2157.301486] Lustre: server umount lustre-MDT0001 complete [ 2161.050579] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2162.143402] Key type lgssc unregistered [ 2162.286369] LNet: 142495:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2162.289722] LNetError: 142495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2162.296322] LNet: Removed LNI 192.168.204.106@tcp [ 2162.594569] Key type .llcrypt unregistered [ 2162.596355] Key type ._llcrypt unregistered [ 2172.104180] Key type ._llcrypt registered [ 2172.105440] Key type .llcrypt registered [ 2172.142238] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2172.461969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2172.504051] alg: No test for adler32 (adler32-zlib) [ 2173.366195] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2173.448717] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2175.032143] Key type lgssc registered [ 2175.432401] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2178.281844] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2178.291817] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2179.474606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2180.862938] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2183.921405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2184.089832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2185.424582] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2187.302873] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2188.701900] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2191.407539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2191.591650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2193.642760] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2193.767444] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2195.936610] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2205.933331] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2219.656662] Lustre: lustre-OST0000: evicting b7a3ac30-846b-4e32-b62d-6cae61b5eab4 (at 192.168.204.6@tcp) by administrative request [ 2219.661399] Lustre: Skipped 1 previous similar message [ 2245.088882] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2245.091859] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2245.096257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2247.137557] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2247.138807] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2247.139556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2247.148222] Lustre: Skipped 1 previous similar message [ 2252.258264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2252.261313] Lustre: Skipped 1 previous similar message [ 2257.377420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2257.381582] Lustre: Skipped 1 previous similar message [ 2258.400920] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2258.458634] Lustre: server umount lustre-OST0000 complete [ 2260.962602] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2260.966463] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2262.497124] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2262.499708] Lustre: Skipped 2 previous similar messages [ 2265.813883] Lustre: server umount lustre-MDT0000 complete [ 2267.130114] LustreError: 143850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441606 with bad export cookie 6393093563725091446 [ 2267.133112] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2267.134217] LustreError: 143850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2267.254724] Lustre: server umount lustre-MDT0001 complete [ 2271.020338] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2272.020470] Key type lgssc unregistered [ 2272.154473] LNet: 147605:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2272.157406] LNetError: 147605:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2272.166606] LNet: Removed LNI 192.168.204.106@tcp [ 2272.476502] Key type .llcrypt unregistered [ 2272.477990] Key type ._llcrypt unregistered [ 2299.282936] Key type ._llcrypt registered [ 2299.284585] Key type .llcrypt registered [ 2299.328730] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2299.771690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2299.785893] alg: No test for adler32 (adler32-zlib) [ 2300.639673] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2300.725876] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2302.304133] Key type lgssc registered [ 2302.654029] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2305.316884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2305.323703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2306.490962] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2307.863859] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2310.901231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2311.040904] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2312.386950] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2314.391762] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2315.698983] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2318.247590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2318.390233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2320.424081] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2320.596422] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2322.613761] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2331.686376] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2336.740618] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2336.741365] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2336.747051] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2336.751063] Lustre: Skipped 1 previous similar message [ 2337.932222] Lustre: server umount lustre-OST0000 complete [ 2341.209922] Lustre: server umount lustre-MDT0000 complete [ 2341.857163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2341.858673] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2341.860975] LustreError: 150051:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2341.864752] Lustre: Skipped 2 previous similar messages [ 2341.871767] LustreError: 150051:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2342.479374] LustreError: 150774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441682 with bad export cookie 15568089275055317606 [ 2342.482439] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2342.483862] LustreError: 150774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2342.599578] Lustre: server umount lustre-MDT0001 complete [ 2346.110284] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2347.114493] Key type lgssc unregistered [ 2347.242371] LNet: 153255:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2347.244656] LNetError: 153255:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2347.255284] LNet: Removed LNI 192.168.204.106@tcp [ 2347.563848] Key type .llcrypt unregistered [ 2347.564889] Key type ._llcrypt unregistered [ 2356.646968] Key type ._llcrypt registered [ 2356.647938] Key type .llcrypt registered [ 2356.684639] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2357.008911] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2357.052074] alg: No test for adler32 (adler32-zlib) [ 2357.917762] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2358.001762] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2359.592263] Key type lgssc registered [ 2359.994576] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2364.669199] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 14:08:23 (1771441703) [ 2369.884475] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 2372.879497] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2385.034238] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2385.039828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2386.154861] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2386.168329] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2386.206584] Lustre: lustre-MDT0000: new disk, initializing [ 2386.236252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2386.248481] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2387.629544] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2391.821345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2391.858396] Lustre: 157436:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2391.880049] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2391.882050] Lustre: Skipped 1 previous similar message [ 2391.913388] Lustre: lustre-MDT0001: new disk, initializing [ 2391.944243] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2391.955952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2391.958905] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2393.173269] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2396.152260] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2397.349582] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2399.522312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2399.621676] Lustre: lustre-OST0000: new disk, initializing [ 2399.623483] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2399.654125] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2401.332661] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2401.405201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2401.409144] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2401.443940] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2404.256169] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2412.002225] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2412.002667] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2412.006217] LustreError: Skipped 1 previous similar message [ 2412.010144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2412.013882] Lustre: Skipped 1 previous similar message [ 2413.455500] Lustre: server umount lustre-OST0000 complete [ 2417.121489] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2417.122308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2417.125072] Lustre: Skipped 2 previous similar messages [ 2417.129129] Lustre: Skipped 1 previous similar message [ 2420.756435] Lustre: server umount lustre-MDT0000 complete [ 2422.026481] LustreError: 157428:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771441761 with bad export cookie 874960342400701001 [ 2422.029831] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2422.031660] LustreError: 157428:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2422.156785] Lustre: server umount lustre-MDT0001 complete [ 2425.686452] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2426.699729] Key type lgssc unregistered [ 2426.821534] LNet: 160949:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2426.825464] LNetError: 160949:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2426.834453] LNet: Removed LNI 192.168.204.106@tcp [ 2427.101375] Key type .llcrypt unregistered [ 2427.102672] Key type ._llcrypt unregistered [ 2435.671121] Key type ._llcrypt registered [ 2435.672122] Key type .llcrypt registered [ 2435.704270] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2436.033817] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2436.063793] alg: No test for adler32 (adler32-zlib) [ 2436.917202] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2436.998995] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2438.576116] Key type lgssc registered [ 2438.923257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2441.369289] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2441.374752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2442.523946] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2443.638049] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2446.253945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2446.370345] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2447.576423] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2449.232779] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2450.354273] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2452.446045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2452.544397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2454.368648] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2456.212994] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2467.940241] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2474.101061] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2474.109143] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2474.113899] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2474.117533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.120555] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2474.437538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.759268] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2474.761444] Lustre: Skipped 1 previous similar message [ 2474.767919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2474.771007] Lustre: Skipped 1 previous similar message [ 2474.772261] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2474.773957] Lustre: Skipped 1 previous similar message [ 2474.775204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.777667] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2474.779436] Lustre: Skipped 1 previous similar message [ 2475.063583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.367354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.663063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.963154] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2475.965264] Lustre: Skipped 3 previous similar messages [ 2475.970899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2475.974279] Lustre: Skipped 3 previous similar messages [ 2475.975466] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2475.977100] Lustre: Skipped 3 previous similar messages [ 2475.978310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.981595] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2475.983397] Lustre: Skipped 3 previous similar messages [ 2476.261831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.547092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.838379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.164170] LustreError: 162342:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2477.167567] LustreError: 162342:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2477.168041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.483729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.789351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.080882] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2478.083131] Lustre: Skipped 6 previous similar messages [ 2478.088170] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2478.092421] Lustre: Skipped 6 previous similar messages [ 2478.094697] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2478.096967] Lustre: Skipped 6 previous similar messages [ 2478.098802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.104485] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2478.106542] Lustre: Skipped 6 previous similar messages [ 2478.396690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.681891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.973539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.280633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.596066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.908576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.222299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.528377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.850139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.156878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.472766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.780324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.107721] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2482.111037] Lustre: Skipped 12 previous similar messages [ 2482.117682] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2482.121493] Lustre: Skipped 12 previous similar messages [ 2482.123191] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2482.125272] Lustre: Skipped 12 previous similar messages [ 2482.126917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.130349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2482.132204] Lustre: Skipped 12 previous similar messages [ 2482.425392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.723604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.021594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.315084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.571579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.837658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.107089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.373069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.655673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.929401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.238696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.570225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.854464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.132232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.399221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.679396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.954664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.233103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.529650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.817857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.091086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.366679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.660324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.970068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.247247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.535664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.823242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.110358] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2490.112811] Lustre: Skipped 27 previous similar messages [ 2490.117597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2490.121657] Lustre: Skipped 27 previous similar messages [ 2490.123219] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2490.125230] Lustre: Skipped 27 previous similar messages [ 2490.126590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.403935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.407724] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2490.409367] Lustre: Skipped 28 previous similar messages [ 2490.681584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.969955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.257854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.540722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.846631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.132229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.434908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.738379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.034384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.324966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.627771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.922327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.205808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.489853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.788199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.086816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.384048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.674064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.971172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.258238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.542327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.827898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.126785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.408842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.699618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.984144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.279593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.545833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.856538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.175986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.490645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.792685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.105056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.390201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.702324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.991241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.281649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.588751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.912254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.199282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.501824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.785527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.084955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.369675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.664945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.954483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.245223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.529763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.813647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.099677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.383764] LustreError: 162342:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2505.386580] LustreError: 162342:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2505.387487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.680251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.989070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.297886] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2506.299781] Lustre: Skipped 54 previous similar messages [ 2506.309677] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2506.314531] Lustre: Skipped 54 previous similar messages [ 2506.316863] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2506.318713] Lustre: Skipped 54 previous similar messages [ 2506.319992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.683545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.686717] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2506.688374] Lustre: Skipped 54 previous similar messages [ 2506.996417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.298587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.585855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.899529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.192160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.508609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.803985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.109675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.399118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.685257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.973236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.261109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.566894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.857501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.165546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.479293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.761302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.047322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.331757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.637055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.934667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.242190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.543810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.855316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.150870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.423023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.705737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.023078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.309958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.597254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.876594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.159417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.431532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.714182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.985422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.268388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.549440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.838639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.127632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.437512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.704663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.990347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.279839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.575062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.852099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.131842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.411060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.695307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.973896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.265384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.553463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.859724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.175840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.458311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.739182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.026299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.299358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.578327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.869538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.175786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.454416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.730749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.019864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.314955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.595312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.881841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.168466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.479781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.778654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.071881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.372444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.699145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.983831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.285144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.578192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.876959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.161321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.440753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.738916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.037462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.342206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.630236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.917975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.232549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.527507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.816954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.118245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.420644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.705250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.994851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.283934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.589664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.880275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.171513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.455720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.762935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.065234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.358222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.661972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.975090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.274811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.565818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.854298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.157605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.452382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.747531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.037257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.341515] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2538.343568] Lustre: Skipped 108 previous similar messages [ 2538.348847] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2538.351760] Lustre: Skipped 108 previous similar messages [ 2538.353037] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2538.354852] Lustre: Skipped 108 previous similar messages [ 2538.356204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.650443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.944400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.948130] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2538.950285] Lustre: Skipped 109 previous similar messages [ 2539.250152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.554129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.846790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.138070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.431823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.739946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.025462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.328476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.637292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.948170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.245984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.546255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.873652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.192239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.477066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.762455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.063975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.363544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.644202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.939076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.227407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.524207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.822837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.120338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.406793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.720027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.014957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.303533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.590263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.898737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.188324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.487367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.782915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.079432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.366325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.667517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.958909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.271622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.550035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.849226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.134256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.434740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.730859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.025779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.316731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.629350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.925614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.214894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.497299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.804450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.096623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.390630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.696765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.003082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.311270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.602859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.921278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.235964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.535489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.838244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.133470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.455702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.753224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.070368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.363870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.672943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.977854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.269960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.553214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.850535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.134453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.406991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.674952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.952217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.234871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.510505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.801493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.094535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.379233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.672436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.945308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.222626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.513257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.794526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.071818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.357796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.650749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.930413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.211176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.496636] LustreError: 162342:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2565.501153] LustreError: 162342:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2565.501494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.806790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.090252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.381883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.666185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.950524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.235459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.525957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.842548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.134292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.425158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.718294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.031345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.340633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.623431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.904283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.215025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.518113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.817811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.106432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.444678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.738141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.022497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.344878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.673956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.969930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.258296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.568889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.886963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.183203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.501667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.795624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.109094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.403645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.695646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.003064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.313361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.613247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.926060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.264237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.616613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.911208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.206866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.501620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.810333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.114176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.442639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.783680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.091901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.389768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.692087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.985169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.277697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.579957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.876213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.174083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.494389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.798928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.094549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.386310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.739209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.039989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.341542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.648853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.956942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.262366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.570588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.881376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.223705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.543925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.871497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.180263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.480027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.781734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.089596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.388722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.703103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.005058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.319225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.620931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.943737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.239069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.534178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.834391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.145425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.453421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.766506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.065446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.379090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.673857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.984876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.283772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.586380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.898909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.196376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.486555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.804489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.099717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.389081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.690609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.013194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.309307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.602924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.886851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.188176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.467522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.758475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.059680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.356948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.624212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.908281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.191963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.491159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.769727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.069926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.343191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.631691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.905324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.181425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.463770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.767552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.041614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.331176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.620357] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2602.622274] Lustre: Skipped 214 previous similar messages [ 2602.627781] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2602.632650] Lustre: Skipped 214 previous similar messages [ 2602.634055] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2602.635785] Lustre: Skipped 214 previous similar messages [ 2602.636958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.930360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.216696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.220249] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2603.222217] Lustre: Skipped 214 previous similar messages [ 2603.510577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.805388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.107630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.385964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.684619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.954821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.260317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.567800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.873894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.189220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.528128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.847462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.169177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.486590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.822235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.127851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.455569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.773726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.090928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.395656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.709838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.036853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.376002] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.681648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.997690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.323422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.675992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.996868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.334919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.658164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.989875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.299353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.605712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.929985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.276443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.610321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.947292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.268361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.606931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.895753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.182075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.467910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.790806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.084976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.375372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.655979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.955906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.245627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.544535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.846279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.129183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.404327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.687690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.964927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.257465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.532609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.810573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.090110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.386444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.654458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.925963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.198680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.479100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.758125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.067328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.351162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.652424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.945187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.233836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.522631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.819411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.099596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.393179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.734954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.062188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.376978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.672362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.949926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.240272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.519753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.804061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.106594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.406739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.712141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.029410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.332519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.643873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.944081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.233462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.520051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.821468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.104478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.383485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.663315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.961555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.253701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.538434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.805459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.103183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.411309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.710373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.014323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.305225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.587072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.872053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.147711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.416917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.687529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.959555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.229477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.510601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.782282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.060494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.354654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.658215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.937788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.228120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.511104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.816632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.107229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.394595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.675294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.985813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.281744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.600582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.886423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.190460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.471422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.762351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.041830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.330242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.638269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.921073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.205322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.506781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.786459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.060643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.327490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.612836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.882548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.182668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.483557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.808609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.122933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.435907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.730275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.049932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.347810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.653031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.969479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.302857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.603923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.905544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.210464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.538129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.828527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.116984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.403802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.722689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.032512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.335469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.628064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.949942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.281631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.593464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.902103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.221273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.517910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.827301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.133443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.448510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.744194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.049973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.343950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.681041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.982024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.289640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.608644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.926605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.225600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.519769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.829346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.155394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.469244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.793377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.108476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.430337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.750515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.071805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.392509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.721448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.030574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.347931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.677381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.018208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.336967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.668055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.996292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.330115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.630406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.915980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.197864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.506532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.796743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.091533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.361776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.633389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.914786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.194567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.469458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.751969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.025262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.298587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.569789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.842752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.113266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.413790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.699234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.976572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.241149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.524597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.797208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.108332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.408144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.736951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.049259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.379776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.716784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.034320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.344757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.662574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.956307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.259489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.545900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.882304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.192724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.498805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.806474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.142171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.459971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.772126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.074755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.390578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.690452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.018431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.342016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.658563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.945400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.253319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.566932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.901782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.228867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.548911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.859397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.193921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.520400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.847852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.160680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.512847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.832729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.137255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.452273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.783830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.097523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.431960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.758809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.078854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.385347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.679313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.999478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.352894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.691341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.039833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.375752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.687500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.983179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.274306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.566259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.845353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.112330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.395561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.688154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.987684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.261667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.550940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.855072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.133770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.413631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.709401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.012696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.304709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.606568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.904328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.204341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.504697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.813469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.122232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.431311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.732174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.035505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.348620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.683127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.012471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.327587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.663060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.991931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.302751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.600741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.908449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.220921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.523140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.821975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.138723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.470591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.810469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.162212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.492686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.817047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.130780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.448053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.790990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.107813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.433023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.748853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.125847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.439882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.766131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.093225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.436146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.756915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.076529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.392347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.737172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.071876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.399437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.692224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.016902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.317599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.624054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.936555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.254235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.575334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.895620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.230428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.579821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.898586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.217148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.533170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.859419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.173271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.484403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.787317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.123906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.443038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.781321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.093045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.459670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.785613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.098457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.409263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.764102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.075863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.424656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.753798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.117454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.447087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.755564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.074323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.402208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.719114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.038917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.357753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.707961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.018553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.347468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.671847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.023171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.347865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.664610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.996319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.341739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.656313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.977986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.291765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.637310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.978735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.308527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.629329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.972895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.300487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.629139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.954492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.280510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.582787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.898371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.219355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.564624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.905942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.239126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.554640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.883123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.193995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.529678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.835591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.159891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.480912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.800617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.141887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.488900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.812153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.140185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.477920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.809531] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2730.812887] Lustre: Skipped 414 previous similar messages [ 2730.820797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2730.825442] Lustre: Skipped 414 previous similar messages [ 2730.827723] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2730.830244] Lustre: Skipped 414 previous similar messages [ 2730.831943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.169057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.507875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.511616] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2731.513631] Lustre: Skipped 414 previous similar messages [ 2731.830349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.188208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.516326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.854429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.178372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.514358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.805730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.102421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.413583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.769843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.086617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.414399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.734053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.064072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.373705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.698901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.031390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.380442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.709694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.037906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.366047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.703962] LustreError: 162342:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2738.709021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.714816] LustreError: 162342:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 2738.718104] LustreError: 162342:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2738.723937] LustreError: 162342:0:(osp_precreate.c:1355:osp_precreate_thread()) Skipped 1 previous similar message [ 2739.052464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.402884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.730122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.047082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.342667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.637207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.924043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.466194] Lustre: server umount lustre-OST0000 complete [ 2768.354745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2768.357266] Lustre: Skipped 1 previous similar message [ 2773.473193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2773.475070] Lustre: Skipped 1 previous similar message [ 2777.629267] Lustre: server umount lustre-MDT0000 complete [ 2778.593227] LustreError: 162322:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2778.601065] LustreError: 162322:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2778.972351] LustreError: 162303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442118 with bad export cookie 13128155973630100058 [ 2778.975979] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2778.979097] LustreError: 162303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2788.375827] Lustre: server umount lustre-MDT0001 complete [ 2792.363750] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2793.496821] Key type lgssc unregistered [ 2793.640373] LNet: 210843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2793.644196] LNetError: 210843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2793.652784] LNet: Removed LNI 192.168.204.106@tcp [ 2793.990517] Key type .llcrypt unregistered [ 2793.991590] Key type ._llcrypt unregistered [ 2803.590918] Key type ._llcrypt registered [ 2803.592685] Key type .llcrypt registered [ 2803.640683] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2804.072768] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2804.081798] alg: No test for adler32 (adler32-zlib) [ 2804.943527] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2805.030298] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2806.608193] Key type lgssc registered [ 2807.012954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2809.797525] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2809.803570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2810.966193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2812.359996] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2815.383778] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2815.522722] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2816.930991] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2818.772790] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2820.107939] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2822.526588] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2822.661766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2824.591030] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2826.503092] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2837.034262] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17013 to 0x280000401:17057) [ 2843.950669] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2844.593728] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2844.601595] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2844.606079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.609507] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2844.609518] LustreError: 212250:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2846.178171] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2846.180559] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2846.184245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2847.192496] Lustre: lustre-OST0000: Not available for connect from 192.168.204.6@tcp (stopping) [ 2847.203766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2851.356927] Lustre: server umount lustre-OST0000 complete [ 2864.688148] LustreError: 212250:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2879.575218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2879.712281] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2881.675625] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2883.453796] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2887.909867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.915456] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2888.075166] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 2889.258408] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b2087790800.ost_server_uuid 50 [ 2889.742926] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b2087790800.ost_server_uuid in FULL state after 0 sec [ 2893.282627] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2893.286647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2893.288324] Lustre: Skipped 2 previous similar messages [ 2896.723470] Lustre: server umount lustre-OST0000 complete [ 2902.497834] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2902.497874] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2902.498545] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2902.504369] Lustre: Skipped 1 previous similar message [ 2907.617152] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2907.619483] Lustre: Skipped 2 previous similar messages [ 2912.224123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2912.297344] Lustre: server umount lustre-MDT0000 complete [ 2912.737221] LustreError: 214384:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2912.742430] LustreError: 214384:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2913.608349] LustreError: 216162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442253 with bad export cookie 18245518966747411575 [ 2913.611701] LustreError: 216162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2913.612311] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2913.736813] Lustre: server umount lustre-MDT0001 complete [ 2917.518426] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2918.600176] Key type lgssc unregistered [ 2918.742436] LNet: 216896:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2918.745040] LNetError: 216896:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2918.752463] LNet: Removed LNI 192.168.204.106@tcp [ 2919.075624] Key type .llcrypt unregistered [ 2919.077494] Key type ._llcrypt unregistered [ 2931.131301] Key type ._llcrypt registered [ 2931.132401] Key type .llcrypt registered [ 2931.174976] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2931.506756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2931.528085] alg: No test for adler32 (adler32-zlib) [ 2932.396147] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 2932.493368] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2934.080176] Key type lgssc registered [ 2934.468821] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2937.625677] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2937.631691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2938.796570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2940.132275] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2943.011788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2943.136832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2944.503600] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2945.463887] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 2947.718791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2947.840415] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2948.905509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17059 to 0x280000401:17089) [ 2950.208506] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2953.056531] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2953.139298] Lustre: lustre-OST0001: new disk, initializing [ 2953.141651] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2953.196988] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2955.118724] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2958.315941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2958.321083] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2958.335255] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2960.630168] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2967.435672] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 2983.909198] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2983.913082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2983.913206] Lustre: Skipped 2 previous similar messages [ 2983.916423] Lustre: Skipped 1 previous similar message [ 2985.889692] Lustre: server umount lustre-MDT0000 complete [ 2987.180713] LustreError: 218565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442326 with bad export cookie 13678108563419469210 [ 2987.184457] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2987.185717] LustreError: 218565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2987.310730] Lustre: server umount lustre-MDT0001 complete [ 2999.233788] Lustre: server umount lustre-OST0000 complete [ 3010.784771] Lustre: server umount lustre-OST0001 complete [ 3017.962649] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3021.593214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3021.745172] LustreError: 223924:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3021.751469] LustreError: 223924:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3021.766909] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3023.073793] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3025.973873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3027.480934] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3028.506086] Lustre: 225029:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3030.839546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3030.979612] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3030.983562] Lustre: Skipped 1 previous similar message [ 3032.035157] LustreError: 225568:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3032.041315] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17092 to 0x280000401:17121) [ 3033.081728] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3036.045073] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3038.036421] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3041.236885] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3042.437883] Lustre: 226872:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3054.947167] Lustre: server umount mylustre-MDT0000 complete [ 3056.259312] LustreError: 223899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442395 with bad export cookie 13678108563419472003 [ 3056.263019] LustreError: 223899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3056.263204] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3056.610040] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3056.611383] LustreError: 225404:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3056.611629] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3056.617325] Lustre: Skipped 4 previous similar messages [ 3058.428614] Lustre: server umount mylustre-MDT0001 complete [ 3059.911961] Lustre: server umount mylustre-OST0000 complete [ 3061.311189] Lustre: server umount mylustre-OST0001 complete [ 3067.770873] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3071.065849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3071.209971] LustreError: 229159:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3071.215885] LustreError: 229159:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3071.229292] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3071.231211] Lustre: Skipped 1 previous similar message [ 3072.441239] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3075.261382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3076.513326] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3077.418730] Lustre: 230264:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3079.495053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3080.612543] LustreError: 230619:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3080.615127] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17123 to 0x280000401:17153) [ 3081.238682] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3084.131735] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3086.024267] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3089.194945] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3090.343639] Lustre: 232107:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3101.153121] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3101.156556] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3101.160715] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3101.163872] Lustre: Skipped 1 previous similar message [ 3103.326647] Lustre: server umount tfs-MDT0000 complete [ 3104.540332] LustreError: 229135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442444 with bad export cookie 13678108563419473669 [ 3104.543676] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3104.544182] LustreError: 229135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3104.656874] Lustre: server umount tfs-MDT0001 complete [ 3115.531875] Lustre: server umount tfs-OST0000 complete [ 3127.076759] Lustre: server umount tfs-OST0001 complete [ 3133.936192] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3137.372120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3137.516329] LustreError: 234339:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3137.535986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3137.537829] Lustre: Skipped 3 previous similar messages [ 3138.831826] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3141.648557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3143.114996] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3144.078319] Lustre: 235446:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3146.374357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3147.496331] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17123 to 0x280000401:17185) [ 3148.396696] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3151.342641] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3153.360095] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3156.486964] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3157.680122] Lustre: 237288:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3164.991364] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 14:21:43 (1771442503) [ 3177.952948] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3177.955779] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3177.958946] Lustre: Skipped 3 previous similar messages [ 3177.961259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3177.963292] Lustre: Skipped 4 previous similar messages [ 3183.586130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3183.587958] Lustre: Skipped 4 previous similar messages [ 3191.264094] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3191.332837] Lustre: server umount lustre-MDT0000 complete [ 3192.550038] LustreError: 234314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442532 with bad export cookie 13678108563419475111 [ 3192.553264] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3192.554802] LustreError: 234314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3193.825073] LustreError: 235984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3193.825772] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3193.826731] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3193.826740] Lustre: Skipped 5 previous similar messages [ 3193.829848] LustreError: 235984:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3193.834320] Lustre: Skipped 4 previous similar messages [ 3198.713359] Lustre: server umount lustre-MDT0001 complete [ 3214.304107] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3214.334587] Lustre: server umount lustre-OST0000 complete [ 3221.655632] Lustre: server umount lustre-OST0001 complete [ 3223.702230] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3226.867788] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3238.695255] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3238.695255] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3238.695255] [ 3238.702176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3238.801086] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3238.815020] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3238.848946] Lustre: lustre-MDT0000: new disk, initializing [ 3238.873628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3238.875570] Lustre: Skipped 3 previous similar messages [ 3238.881167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3240.172854] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3244.314923] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3244.314923] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3244.314923] [ 3244.323273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3244.363533] Lustre: 240431:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3244.385895] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3244.387570] Lustre: Skipped 1 previous similar message [ 3244.421372] Lustre: lustre-MDT0001: new disk, initializing [ 3244.467684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3244.472114] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3245.796803] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3249.968537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3250.074828] Lustre: lustre-OST0000: new disk, initializing [ 3250.077080] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3251.387648] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3251.392260] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3251.424550] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3252.023789] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3256.207491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3256.283714] Lustre: lustre-OST0001: new disk, initializing [ 3256.286973] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3257.359856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3257.364220] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3257.410797] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3258.216141] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3262.692075] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 14:23:21 (1771442601) [ 3266.529573] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 14:23:25 (1771442605) [ 3282.401145] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3282.401222] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3282.403978] LustreError: Skipped 1 previous similar message [ 3282.407964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3282.410696] Lustre: Skipped 4 previous similar messages [ 3284.029884] Lustre: server umount lustre-OST0000 complete [ 3285.472836] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3288.034625] LustreError: 241965:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3288.039490] LustreError: 241965:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3291.617765] Lustre: server umount lustre-MDT0000 complete [ 3292.897479] LustreError: 241208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442632 with bad export cookie 13678108563419476434 [ 3292.901970] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3299.054861] Lustre: server umount lustre-MDT0001 complete [ 3303.066532] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3321.312150] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3321.374604] Lustre: server umount lustre-OST0001 complete [ 3323.399322] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3329.641404] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3341.490962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3341.592622] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3341.603457] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3341.644866] Lustre: lustre-MDT0000: new disk, initializing [ 3341.675667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3342.909606] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3347.197090] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3347.239193] Lustre: 247294:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3347.342971] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3348.608481] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3351.764844] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3353.066352] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3355.430413] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3356.990442] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3357.033388] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3357.479800] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3360.552645] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3367.395501] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3367.396233] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3367.397093] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3367.397098] Lustre: Skipped 6 previous similar messages [ 3367.398454] LustreError: Skipped 1 previous similar message [ 3367.407334] Lustre: Skipped 6 previous similar messages [ 3368.976642] Lustre: server umount lustre-OST0000 complete [ 3376.350294] Lustre: server umount lustre-MDT0000 complete [ 3377.577106] LustreError: 247286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442717 with bad export cookie 13678108563419477911 [ 3377.580500] LustreError: 247286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3377.581048] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3377.700335] Lustre: server umount lustre-MDT0001 complete [ 3381.562251] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3382.605137] Key type lgssc unregistered [ 3382.733355] LNet: 250660:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3382.735905] LNetError: 250660:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3382.744306] LNet: Removed LNI 192.168.204.106@tcp [ 3383.032156] Key type .llcrypt unregistered [ 3383.034072] Key type ._llcrypt unregistered [ 3393.192309] Key type ._llcrypt registered [ 3393.193438] Key type .llcrypt registered [ 3393.226283] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3393.666186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3393.676132] alg: No test for adler32 (adler32-zlib) [ 3394.524244] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 3394.604470] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3396.184122] Key type lgssc registered [ 3396.543645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3399.110060] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3399.117340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3400.274120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3401.519489] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3404.354138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3404.470215] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3405.746015] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3407.505334] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3408.762452] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3411.012046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3411.125234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3413.017873] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3414.741226] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3415.067334] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3420.642064] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3420.642238] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3420.648117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3421.773607] Lustre: server umount lustre-OST0000 complete [ 3423.002153] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3423.002890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3423.005818] Lustre: Skipped 1 previous similar message [ 3423.008864] Lustre: Skipped 1 previous similar message [ 3426.016973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3426.018675] Lustre: Skipped 1 previous similar message [ 3429.143850] Lustre: server umount lustre-MDT0000 complete [ 3430.345130] LustreError: 252064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442770 with bad export cookie 17718839863370348754 [ 3430.348971] LustreError: 252064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3430.348976] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3430.458542] Lustre: server umount lustre-MDT0001 complete [ 3434.356841] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3435.491609] Key type lgssc unregistered [ 3435.618397] LNet: 255308:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3435.622491] LNetError: 255308:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3435.633242] LNet: Removed LNI 192.168.204.106@tcp [ 3435.914349] Key type .llcrypt unregistered [ 3435.915351] Key type ._llcrypt unregistered [ 3442.437144] Key type ._llcrypt registered [ 3442.438535] Key type .llcrypt registered [ 3442.489217] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3442.792543] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3442.798522] alg: No test for adler32 (adler32-zlib) [ 3443.659506] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 3443.756431] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3445.336129] Key type lgssc registered [ 3445.692245] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3448.818812] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3448.827014] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3464.288240] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3469.408143] LustreError: 256461:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.106@tcp: failed processing log, type 4: rc = -110 [ 3496.032220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3498.080615] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3499.936987] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3500.849813] Lustre: server umount lustre-OST0000 complete [ 3507.192824] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3508.480920] Key type lgssc unregistered [ 3508.593303] LNet: 257960:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3508.595578] LNetError: 257960:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3508.602285] LNet: Removed LNI 192.168.204.106@tcp [ 3508.869339] Key type .llcrypt unregistered [ 3508.870376] Key type ._llcrypt unregistered [ 3523.982905] Key type ._llcrypt registered [ 3523.984427] Key type .llcrypt registered [ 3524.022502] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3524.441148] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3524.472023] alg: No test for adler32 (adler32-zlib) [ 3525.334618] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 3525.417033] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3527.000238] Key type lgssc registered [ 3527.438874] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3532.533262] loop0: detected capacity change from 0 to 204800000 [ 3532.611103] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3532.616525] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3532.764722] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3533.124413] loop0: detected capacity change from 0 to 204800000 [ 3533.179486] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3533.183864] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3533.304174] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3533.663167] loop0: detected capacity change from 0 to 204800000 [ 3533.743146] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3533.747578] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3533.804913] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3534.301119] loop0: detected capacity change from 0 to 204800000 [ 3534.393393] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3534.931288] loop0: detected capacity change from 0 to 204800000 [ 3534.947205] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3536.412395] loop0: detected capacity change from 0 to 204800000 [ 3536.422758] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3537.892782] loop0: detected capacity change from 0 to 204800000 [ 3537.902663] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3539.319225] loop0: detected capacity change from 0 to 204800000 [ 3539.329228] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3540.632587] loop0: detected capacity change from 0 to 204800000 [ 3540.662260] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3540.671116] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3540.711063] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3540.717265] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3540.721551] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3540.726833] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3540.731298] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3540.735595] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3540.739504] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3540.743154] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3540.747200] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3540.751095] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3540.756066] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3540.759782] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3540.763681] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3540.767301] Lustre: 261331:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3543.380788] Lustre: server umount MGS complete [ 3543.736154] loop0: detected capacity change from 0 to 204800000 [ 3543.765488] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3543.941524] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3543.961211] Lustre: lustre-MDT0000: Aborting client recovery [ 3544.324460] loop1: detected capacity change from 0 to 204800000 [ 3544.394938] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3544.423849] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3544.428169] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3544.431969] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3544.436106] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3544.440211] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3544.444584] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3544.450207] Lustre: 261761:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3544.458384] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3544.533501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3544.570775] Lustre: lustre-MDT0001: Aborting client recovery [ 3544.924273] loop2: detected capacity change from 0 to 204800000 [ 3544.958952] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3544.994187] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3544.998149] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3545.004073] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3545.010538] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3545.014468] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3545.018129] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3545.022965] Lustre: 261841:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3545.079406] Lustre: Found index 0 for lustre-OST0000, updating log [ 3545.115313] Lustre: lustre-OST0000: Aborting recovery [ 3545.500571] loop3: detected capacity change from 0 to 204800000 [ 3545.548498] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3545.592794] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3545.598374] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3545.601920] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3545.605711] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3545.609182] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3545.612622] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3545.616732] Lustre: 261925:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3545.652362] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3545.655266] Lustre: Skipped 1 previous similar message [ 3545.663875] Lustre: lustre-OST0001: Aborting recovery [ 3550.698893] LustreError: 261705:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 3553.765329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3553.766295] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3553.782556] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3553.787126] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3554.787715] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3554.787730] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3556.287614] LustreError: 261673:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 282us: rc = -2 [ 3570.146119] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3570.146801] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3570.147478] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3570.150591] Lustre: Skipped 1 previous similar message [ 3573.802411] Lustre: server umount lustre-MDT0000 complete [ 3574.115837] LustreError: 261656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771442913 with bad export cookie 10472466468689019806 [ 3574.117515] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3574.119778] LustreError: 261656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3575.264687] LustreError: 261669:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3575.264897] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3575.265119] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3575.265125] Lustre: Skipped 3 previous similar messages [ 3575.272691] LustreError: 261669:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3575.275961] Lustre: Skipped 3 previous similar messages [ 3579.296467] LustreError: 261669:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3579.296574] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3579.301805] LustreError: 261669:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3579.303638] Lustre: Skipped 2 previous similar messages [ 3580.389773] Lustre: server umount lustre-MDT0001 complete [ 3595.232223] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3595.304711] Lustre: server umount lustre-OST0000 complete [ 3601.821599] Lustre: server umount lustre-OST0001 complete [ 3604.544719] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 14:29:03 (1771442943) [ 3615.753219] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3619.153365] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3631.916451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3632.003196] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3632.015759] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3632.052624] Lustre: lustre-MDT0000: new disk, initializing [ 3632.075602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3632.082111] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3633.459841] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3637.836923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3637.890946] Lustre: 265965:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3637.916350] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3637.918122] Lustre: Skipped 1 previous similar message [ 3637.958659] Lustre: lustre-MDT0001: new disk, initializing [ 3637.993677] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3638.009352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3638.013773] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3639.250917] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3642.317039] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3643.771254] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3646.314349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3646.445706] Lustre: lustre-OST0000: new disk, initializing [ 3646.448222] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3646.486598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3648.664373] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3649.473376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3649.478076] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3649.522651] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3652.217138] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3654.619880] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3659.063805] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3666.522274] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3675.257058] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3716.066860] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3716.067495] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3716.071183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3716.073145] LustreError: Skipped 1 previous similar message [ 3721.186863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3721.188965] Lustre: Skipped 2 previous similar messages [ 3727.328110] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3727.398678] Lustre: server umount lustre-OST0000 complete [ 3730.400814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3730.403243] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3730.406931] Lustre: Skipped 1 previous similar message [ 3730.408498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3730.410368] Lustre: Skipped 2 previous similar messages [ 3743.200110] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3743.258599] Lustre: server umount lustre-MDT0000 complete [ 3744.380928] LustreError: 265957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443084 with bad export cookie 10472466468689022515 [ 3744.383988] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3744.384765] LustreError: 265957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3744.497294] Lustre: server umount lustre-MDT0001 complete [ 3747.108730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3748.313303] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3749.845986] Lustre: server umount MGS complete [ 3753.362914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3753.489337] LustreError: 271498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3753.495515] LustreError: 271510:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3753.513760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3754.686245] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3757.484682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3758.931808] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3760.792111] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3761.996760] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3764.263674] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3766.288645] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3768.057663] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3808.738586] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3808.740958] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3808.744517] Lustre: Skipped 1 previous similar message [ 3808.746287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3808.749203] Lustre: Skipped 6 previous similar messages [ 3811.402524] Lustre: server umount lustre-OST0000 complete [ 3813.856891] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3813.858983] LustreError: Skipped 1 previous similar message [ 3818.708174] Lustre: server umount lustre-MDT0000 complete [ 3819.863128] LustreError: 271477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443159 with bad export cookie 10472466468689024440 [ 3819.865577] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3819.866234] LustreError: 271477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3819.981236] Lustre: server umount lustre-MDT0001 complete [ 3823.286048] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 3824.265266] Key type lgssc unregistered [ 3824.388400] LNet: 275108:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3824.390766] LNetError: 275108:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3824.400276] LNet: Removed LNI 192.168.204.106@tcp [ 3824.658901] Key type .llcrypt unregistered [ 3824.659967] Key type ._llcrypt unregistered [ 3836.236113] Key type ._llcrypt registered [ 3836.237289] Key type .llcrypt registered [ 3836.273319] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3842.399464] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3842.715086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3842.757926] alg: No test for adler32 (adler32-zlib) [ 3843.612567] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 3843.696036] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 3845.272160] Key type lgssc registered [ 3845.622686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3856.784298] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3856.789339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3857.874640] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3857.885588] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3857.921050] Lustre: lustre-MDT0000: new disk, initializing [ 3857.943792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3857.949939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3859.232277] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3863.290505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3863.334832] Lustre: 279002:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3863.356780] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3863.358792] Lustre: Skipped 1 previous similar message [ 3863.394143] Lustre: lustre-MDT0001: new disk, initializing [ 3863.426328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3863.438796] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3863.441930] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3864.675597] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3867.696211] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3868.893620] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3871.081261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3871.179707] Lustre: lustre-OST0000: new disk, initializing [ 3871.181483] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3871.211612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3873.019466] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3875.833539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3875.836407] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3875.869084] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3876.019156] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3879.356348] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3886.906586] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3892.677196] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3901.289164] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3937.251615] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3937.251675] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3937.252672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3937.255380] Lustre: Skipped 1 previous similar message [ 3937.257754] LustreError: Skipped 1 previous similar message [ 3940.377561] Lustre: server umount lustre-OST0000 complete [ 3942.369825] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3942.370503] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3942.373474] Lustre: Skipped 1 previous similar message [ 3942.378822] Lustre: Skipped 2 previous similar messages [ 3947.489106] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3947.491472] Lustre: Skipped 1 previous similar message [ 3947.742539] Lustre: server umount lustre-MDT0000 complete [ 3948.959129] LustreError: 278995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443288 with bad export cookie 13595385572383751894 [ 3948.962132] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3948.963062] LustreError: 278995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3949.082631] Lustre: server umount lustre-MDT0001 complete [ 3952.015410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3953.404043] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3954.996796] Lustre: server umount MGS complete [ 3958.711435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3958.887855] LustreError: 284626:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3958.911371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3960.231487] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3963.122398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3963.252781] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3964.626248] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3966.445513] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3967.610834] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3969.795976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3971.776985] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3973.565211] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4008.928891] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4008.933150] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4008.940095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4009.954548] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4012.313118] Lustre: server umount lustre-OST0000 complete [ 4014.561464] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4014.566108] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4014.573628] Lustre: Skipped 1 previous similar message [ 4014.577806] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4014.583616] Lustre: Skipped 2 previous similar messages [ 4019.743152] Lustre: server umount lustre-MDT0000 complete [ 4020.193068] LustreError: 285325:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4020.201567] LustreError: 285325:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4021.025299] LustreError: 287023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443360 with bad export cookie 13595385572383753721 [ 4021.028500] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4021.029344] LustreError: 287023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4021.147656] Lustre: server umount lustre-MDT0001 complete [ 4024.831082] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4025.835834] Key type lgssc unregistered [ 4025.962287] LNet: 288223:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4025.964876] LNetError: 288223:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4025.974276] LNet: Removed LNI 192.168.204.106@tcp [ 4026.250890] Key type .llcrypt unregistered [ 4026.252295] Key type ._llcrypt unregistered [ 4033.284159] Key type ._llcrypt registered [ 4033.285222] Key type .llcrypt registered [ 4033.324371] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 4039.067600] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4039.365833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4039.393200] alg: No test for adler32 (adler32-zlib) [ 4040.238051] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 4040.323759] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4041.904220] Key type lgssc registered [ 4042.265085] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4050.665141] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4055.882502] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4055.889615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4056.979949] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4056.989666] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4057.023635] Lustre: lustre-MDT0000: new disk, initializing [ 4057.046403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4057.052148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4058.517604] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4063.447162] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4063.610687] Lustre: lustre-OST0000: new disk, initializing [ 4063.614927] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4063.618921] Lustre: Skipped 1 previous similar message [ 4063.660608] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4066.007675] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4066.580679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4066.587355] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4066.603536] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4286.945941] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4286.952556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4288.416532] Lustre: server umount lustre-MDT0000 complete [ 4289.750352] LustreError: 291380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443629 with bad export cookie 4875013023712911653 [ 4289.752218] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4289.755422] LustreError: 291380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4289.810208] Lustre: server umount lustre-OST0000 complete [ 4298.216896] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 4301.533172] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4313.824845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4313.912929] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4313.924255] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4313.959447] Lustre: lustre-MDT0000: new disk, initializing [ 4313.983114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4313.990093] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4315.314641] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4319.348379] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4319.389735] Lustre: 296358:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4319.411576] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4319.413767] Lustre: Skipped 1 previous similar message [ 4319.445924] Lustre: lustre-MDT0001: new disk, initializing [ 4319.476346] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4319.487563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4319.492982] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4320.884720] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4323.970257] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4325.223698] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4327.648237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4327.748463] Lustre: lustre-OST0000: new disk, initializing [ 4327.750268] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4327.777481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4329.274706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4329.278381] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4329.318754] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4329.659527] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4332.749961] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4334.054322] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4334.130531] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4335.594559] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4335.678690] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4339.683124] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4339.683618] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4339.686145] LustreError: Skipped 1 previous similar message [ 4339.690636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4339.693652] Lustre: Skipped 1 previous similar message [ 4342.491071] Lustre: server umount lustre-OST0000 complete [ 4344.801627] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4344.802237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4344.805108] Lustre: Skipped 2 previous similar messages [ 4349.921017] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4349.923213] Lustre: Skipped 2 previous similar messages [ 4355.041444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4355.044260] Lustre: Skipped 1 previous similar message [ 4358.112145] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4358.174773] Lustre: server umount lustre-MDT0000 complete [ 4359.459647] LustreError: 296350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443699 with bad export cookie 4875013023715852437 [ 4359.463680] LustreError: 296350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4359.463792] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4359.583454] Lustre: server umount lustre-MDT0001 complete [ 4360.746917] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4361.281370] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 14:41:40 (1771443700) [ 4363.565976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4363.738935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4365.034904] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4368.082713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4369.553221] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4371.461427] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4372.697942] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4375.123571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4377.246216] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4379.219068] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4381.826913] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4381.895357] Lustre: lustre-OST0001: new disk, initializing [ 4381.897382] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4381.927038] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4381.929593] Lustre: Skipped 2 previous similar messages [ 4383.928505] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4386.816093] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4386.819970] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4386.835037] LustreError: 303313:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4386.839273] LustreError: 303313:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4387.875335] LustreError: 303313:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4387.878844] LustreError: 303313:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4388.904949] LustreError: 303319:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4388.910313] LustreError: 303319:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4390.947260] LustreError: 303319:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4390.950665] LustreError: 303319:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4393.378631] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4393.464117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4394.911181] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b2086cc9000.ost_server_uuid 50 [ 4395.043515] LustreError: 303313:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4395.047279] LustreError: 303313:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 4395.504984] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b2086cc9000.ost_server_uuid in FULL state after 0 sec [ 4396.860487] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8b2086cc9000.ost_server_uuid 50 [ 4397.417187] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8b2086cc9000.ost_server_uuid in FULL state after 0 sec [ 4398.120791] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4412.381258] Lustre: lustre-OST0001: Not available for connect from 192.168.204.6@tcp (stopping) [ 4413.410201] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4413.411262] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4413.418050] LustreError: Skipped 1 previous similar message [ 4415.493619] Lustre: server umount lustre-OST0001 complete [ 4418.528887] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4418.531902] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4418.536975] Lustre: Skipped 1 previous similar message [ 4419.477507] Lustre: server umount lustre-OST0000 complete [ 4424.673233] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4429.793062] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4429.795687] Lustre: Skipped 8 previous similar messages [ 4430.846858] Lustre: server umount lustre-MDT0000 complete [ 4432.232289] LustreError: 300243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443771 with bad export cookie 4875013023715853305 [ 4432.235823] LustreError: 300243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4432.235994] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4432.369316] Lustre: server umount lustre-MDT0001 complete [ 4436.258603] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4437.358112] Key type lgssc unregistered [ 4437.496496] LNet: 305202:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4437.499660] LNetError: 305202:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4437.514967] LNet: Removed LNI 192.168.204.106@tcp [ 4437.899271] Key type .llcrypt unregistered [ 4437.900780] Key type ._llcrypt unregistered [ 4447.557908] Key type ._llcrypt registered [ 4447.559119] Key type .llcrypt registered [ 4447.595865] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4448.046434] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4448.062652] alg: No test for adler32 (adler32-zlib) [ 4448.920095] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 4449.005466] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4450.592173] Key type lgssc registered [ 4450.974237] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4453.752556] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4453.759208] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4454.941996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4456.298526] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4458.222740] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4460.588060] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4460.723043] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4462.085597] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4465.291039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4465.406391] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4467.277155] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4469.149531] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4470.564641] LustreError: 308132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4470.573023] LustreError: 308132:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4471.637911] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4471.720410] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4473.781712] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4475.691592] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4479.534634] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4479.539056] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4483.429183] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4483.503406] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4560.353120] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4560.356040] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4560.362686] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4561.382284] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4561.383702] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4561.383815] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4566.219319] Lustre: server umount lustre-OST0001 complete [ 4566.501046] LustreError: 308852:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4566.507800] LustreError: 308852:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4568.032870] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4568.035659] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4568.039933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4568.041953] Lustre: Skipped 1 previous similar message [ 4571.617705] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4571.619677] LustreError: 308131:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4571.620532] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4571.621101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4571.634784] LustreError: 308131:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4576.738334] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4576.738414] LustreError: 309574:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4576.743041] Lustre: Skipped 2 previous similar messages [ 4576.750135] LustreError: 309574:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4581.857904] LustreError: 309574:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4581.864413] LustreError: 309574:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4582.368169] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4582.422391] Lustre: server umount lustre-OST0000 complete [ 4583.905050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4583.907638] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4586.977193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4586.979697] Lustre: Skipped 4 previous similar messages [ 4589.792722] Lustre: server umount lustre-MDT0000 complete [ 4591.055484] LustreError: 309899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771443930 with bad export cookie 2230389455164698853 [ 4591.058327] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4591.060588] LustreError: 309899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4591.180762] Lustre: server umount lustre-MDT0001 complete [ 4594.779340] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4595.824469] Key type lgssc unregistered [ 4595.951677] LNet: 311136:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4595.956659] LNetError: 311136:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4595.967293] LNet: Removed LNI 192.168.204.106@tcp [ 4596.282229] Key type .llcrypt unregistered [ 4596.283874] Key type ._llcrypt unregistered [ 4615.285421] Key type ._llcrypt registered [ 4615.286707] Key type .llcrypt registered [ 4615.324361] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4615.659021] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4615.705959] alg: No test for adler32 (adler32-zlib) [ 4616.566074] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 4616.644391] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4618.224105] Key type lgssc registered [ 4618.593225] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4621.762171] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4621.768583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4622.932408] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4624.272696] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4627.394353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4627.540056] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4628.928288] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4629.943499] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4632.206332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4632.320460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4633.379812] LustreError: 314840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4633.385265] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4633.386481] LustreError: 314840:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4634.235727] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4637.169341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4637.265490] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4638.321349] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4639.257117] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4643.527501] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4648.276511] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4658.144842] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4658.147558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4658.152174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4659.172581] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4659.172757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4659.176872] Lustre: Skipped 1 previous similar message [ 4659.178895] Lustre: Skipped 1 previous similar message [ 4663.010461] Lustre: server umount lustre-MDT0000 complete [ 4664.275203] LustreError: 316331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771444003 with bad export cookie 11654130677978992276 [ 4664.279036] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4664.282813] LustreError: 316331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4664.290113] LustreError: 313380:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4664.290389] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4664.290395] Lustre: Skipped 1 previous similar message [ 4664.291202] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4664.296618] LustreError: 313380:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4664.308076] Lustre: Skipped 3 previous similar messages [ 4664.404339] Lustre: server umount lustre-MDT0001 complete [ 4671.941102] Lustre: server umount lustre-OST0000 complete [ 4673.406078] Lustre: server umount lustre-OST0001 complete [ 4679.298684] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4683.313966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4683.459178] LustreError: 318568:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4683.481709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4684.889311] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4688.174392] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4689.817722] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4690.952526] Lustre: 319673:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4693.396761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4693.475177] LustreError: 320030:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4693.483400] LustreError: 320030:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4693.538372] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4693.541312] Lustre: Skipped 1 previous similar message [ 4695.413573] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4695.583846] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4698.468989] LustreError: 320040:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4698.474341] LustreError: 320040:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4698.911242] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4700.856047] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4701.047706] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4704.206561] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4705.405447] Lustre: 321513:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4713.953138] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4713.957218] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4713.962121] Lustre: Skipped 1 previous similar message [ 4713.964397] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4717.534032] Lustre: server umount lustre-MDT0000 complete [ 4718.840623] LustreError: 321516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771444058 with bad export cookie 11654130677978993935 [ 4718.843785] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4718.845376] LustreError: 321516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4718.971326] Lustre: server umount lustre-MDT0001 complete [ 4729.480911] Lustre: server umount lustre-OST0000 complete [ 4741.095355] Lustre: server umount lustre-OST0001 complete [ 4742.267862] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 4742.817235] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 4743.399544] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 14:48:02 (1771444082) [ 4750.879331] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4754.052105] loop0: detected capacity change from 0 to 1099511627776 [ 4754.226195] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 4754.229747] *** VALIDATE xfs *** [ 4754.234622] XFS (loop0): Mounting V5 Filesystem [ 4754.245581] XFS (loop0): Ending clean mount [ 4754.741583] loop1: detected capacity change from 0 to 18691697672192 [ 4775.125041] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 4778.519364] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4790.088447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4790.180824] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4790.193623] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4790.232660] Lustre: lustre-MDT0000: new disk, initializing [ 4790.263406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4790.265579] Lustre: Skipped 1 previous similar message [ 4790.272191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4791.676581] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4796.026927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4796.066755] Lustre: 326576:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4796.089895] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4796.092076] Lustre: Skipped 1 previous similar message [ 4796.126938] Lustre: lustre-MDT0001: new disk, initializing [ 4796.174695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4796.178227] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4797.395206] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4800.549419] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4801.935342] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4804.314860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4804.426119] Lustre: lustre-OST0000: new disk, initializing [ 4804.428667] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4805.823366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4805.827574] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4805.876691] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4806.535871] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4809.778617] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4811.178693] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4811.253874] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4812.562746] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4812.630987] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4816.353927] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4816.355854] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4816.356738] LustreError: Skipped 1 previous similar message [ 4816.360233] Lustre: Skipped 3 previous similar messages [ 4816.362246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4816.364726] Lustre: Skipped 5 previous similar messages [ 4819.358178] Lustre: server umount lustre-OST0000 complete [ 4826.593107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4826.595791] Lustre: Skipped 3 previous similar messages [ 4826.775615] Lustre: server umount lustre-MDT0000 complete [ 4827.982193] LustreError: 328314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771444167 with bad export cookie 11654130677978995489 [ 4827.985777] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4827.986387] LustreError: 328314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4828.099440] Lustre: server umount lustre-MDT0001 complete [ 4830.899807] XFS (loop0): Unmounting Filesystem [ 4832.689750] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 14:49:31 (1771444171) [ 4835.041735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4835.201372] LustreError: 330728:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4835.231054] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4835.233880] Lustre: Skipped 2 previous similar messages [ 4836.608894] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4839.562287] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4840.997487] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4842.747244] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4843.881337] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4845.995365] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4847.907462] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4849.630804] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4864.482298] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4864.483337] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4864.485686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4864.490015] Lustre: Skipped 3 previous similar messages [ 4865.678030] Lustre: server umount lustre-OST0000 complete [ 4873.052063] Lustre: server umount lustre-MDT0000 complete [ 4874.322263] LustreError: 330707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771444214 with bad export cookie 11654130677978996371 [ 4874.326033] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4874.326919] LustreError: 330707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4874.436741] Lustre: server umount lustre-MDT0001 complete [ 4877.933630] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4878.970389] Key type lgssc unregistered [ 4879.099371] LNet: 334021:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4879.101842] LNetError: 334021:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4879.109300] LNet: Removed LNI 192.168.204.106@tcp [ 4879.418122] Key type .llcrypt unregistered [ 4879.419519] Key type ._llcrypt unregistered [ 4889.659290] Key type ._llcrypt registered [ 4889.660594] Key type .llcrypt registered [ 4889.697926] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4890.015149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4890.035096] alg: No test for adler32 (adler32-zlib) [ 4890.884726] Lustre: Lustre: Build Version: 2.17.50_152_g0212678 [ 4890.965561] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4892.544177] Key type lgssc registered [ 4892.893556] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4895.421612] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4895.428177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4896.580271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4897.873207] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4900.757506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4900.897291] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4902.214543] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4904.048332] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4905.264663] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4907.483166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4907.590561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4909.354882] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4911.214536] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4931.552796] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 4931.555195] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4931.559592] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4932.577239] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4932.579326] Lustre: Skipped 1 previous similar message [ 4932.579512] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4935.501385] Lustre: server umount lustre-MDT0001 complete [ 4937.168901] LustreError: 335543:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4937.175067] LustreError: 335543:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4937.698160] LustreError: 335546:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4937.703870] LustreError: 335546:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4938.865106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4938.898895] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 4938.900925] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4938.913481] Lustre: 335535:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4939.023728] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4940.422543] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4944.357847] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 4945.382184] LustreError: 338932:0:(obd_mount.c:212:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 4945.385584] LustreError: 338932:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 4945.389146] LustreError: 335538:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 4945.406970] LustreError: 338935:0:(obd_config.c:2136:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -17 [ 4945.410561] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 4945.410561] [ 5244.375604] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [ 5244.378408] Lustre: Skipped 1 previous similar message [ 5246.434821] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5246.434912] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5246.437469] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5246.440786] Lustre: Skipped 1 previous similar message [ 5248.339624] Lustre: server umount lustre-MDT0001 complete [ 5249.488715] LustreError: 337701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5251.554607] LustreError: 337712:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5251.560615] LustreError: 337712:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5251.695828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5251.728370] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5251.731199] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5251.745402] Lustre: 335536:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5251.847297] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5253.134471] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5257.191374] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5259.750438] LustreError: 351639:0:(obd_mount.c:212:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5259.753419] LustreError: 351639:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5259.756923] LustreError: 335538:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5259.759815] LustreError: 335538:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5259.776865] LustreError: 351642:0:(obd_config.c:797:class_setup()) Device 8 already setup (type osp) [ 5259.779070] LustreError: 351642:0:(obd_config.c:902:class_detach()) OBD device 8 still set up [ 5259.781432] LustreError: 351642:0:(obd_config.c:2136:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -17 [ 5259.786066] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.106@tcp [ 5259.786066] [ 5556.696286] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [ 5556.699711] Lustre: Skipped 1 previous similar message [ 5559.268967] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5559.269069] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5562.584583] Lustre: server umount lustre-MDT0001 complete [ 5564.385817] LustreError: 337646:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5564.391483] LustreError: 337646:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5566.084844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5566.119778] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5566.121680] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5566.134221] Lustre: 335535:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5566.251860] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5567.560954] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5571.557817] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5573.095518] LustreError: 364206:0:(obd_mount.c:212:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5573.098351] LustreError: 364206:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5573.101450] LustreError: 335538:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5573.103961] LustreError: 335538:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5573.122977] LustreError: 364209:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5573.127819] LustreError: 364209:0:(obd_config.c:2136:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -114 [ 5573.130608] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5573.130608] [ 5883.874250] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5883.875512] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5883.881203] Lustre: Skipped 1 previous similar message [ 5883.883278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5883.885512] Lustre: Skipped 5 previous similar messages [ 5888.088837] Lustre: server umount lustre-MDT0000 complete [ 5888.994360] LustreError: 337054:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5889.006680] LustreError: 337054:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5889.395253] LustreError: 337162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771445229 with bad export cookie 8780679639624197915 [ 5889.398412] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5889.399177] LustreError: 337162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5894.113392] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5894.117322] Lustre: Skipped 2 previous similar messages [ 5895.603547] Lustre: server umount lustre-MDT0001 complete [ 5897.029026] Lustre: server umount lustre-OST0000 complete [ 5900.037729] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 5903.188218] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 5915.229125] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:07:34 (1771445254) [ 5917.464419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5917.570730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5917.583700] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5917.623874] Lustre: lustre-MDT0000: new disk, initializing [ 5917.651876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5917.659609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5919.039817] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5923.148092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5923.191111] Lustre: 378736:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5923.214821] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5923.216954] Lustre: Skipped 1 previous similar message [ 5923.251569] Lustre: lustre-MDT0001: new disk, initializing [ 5923.284778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5923.298053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5923.301239] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5924.613537] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5927.753042] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5928.917359] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5931.171665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5931.274938] Lustre: lustre-OST0000: new disk, initializing [ 5931.276880] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5931.304146] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5931.676308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5931.680557] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5931.705964] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5933.208123] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5936.249149] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5942.243339] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5942.245359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5942.249116] Lustre: Skipped 2 previous similar messages [ 5942.253678] Lustre: Skipped 3 previous similar messages [ 5944.987576] Lustre: server umount lustre-MDT0000 complete [ 5946.202444] LustreError: 378728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1771445285 with bad export cookie 8780679639624199854 [ 5946.204480] LustreError: 379525:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 5946.204937] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5946.207367] LustreError: 378728:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5946.327712] Lustre: server umount lustre-MDT0001 complete [ 5957.920655] Lustre: server umount lustre-OST0000 complete [ 5967.328788] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 5970.778638] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 5981.266397] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 5851 sec ========== 15:08:40 (1771445320) [ 5981.771590] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:08:40 (1771445320) === [ 5982.899718] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:08:41 (1771445321) === [ 5995.968361] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 5997.057146] Key type lgssc unregistered [ 5997.196410] LNet: 386666:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5997.200288] LNetError: 386666:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5997.213442] LNet: Removed LNI 192.168.204.106@tcp [ 5997.581343] Key type .llcrypt unregistered [ 5997.583044] Key type ._llcrypt unregistered