[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 570233835 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.996 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2576MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002606] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004011] kvm-guest: setup PV IPIs [ 0.007502] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229833f6470, max_idle_ns: 440795327230 ns [ 0.008019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399996) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010306] LSM: Security Framework initializing [ 0.011051] Yama: becoming mindful. [ 0.011861] SELinux: Initializing. [ 0.012070] *** VALIDATE selinux *** [ 0.021690] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026193] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028041] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029126] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030141] *** VALIDATE tmpfs *** [ 0.031469] *** VALIDATE proc *** [ 0.033102] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035274] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036156] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038029] Spectre V2 : User space: Vulnerable [ 0.039007] Speculative Store Bypass: Vulnerable [ 0.041871] debug: unmapping init [mem 0xffffffffb3a59000-0xffffffffb3a60fff] [ 0.044000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044942] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046024] ... version: 2 [ 0.047009] ... bit width: 48 [ 0.048017] ... generic registers: 4 [ 0.049014] ... value mask: 0000ffffffffffff [ 0.050011] ... max period: 00007fffffffffff [ 0.051016] ... fixed-purpose events: 3 [ 0.052010] ... event mask: 000000070000000f [ 0.053408] rcu: Hierarchical SRCU implementation. [ 0.055684] smp: Bringing up secondary CPUs ... [ 0.056843] x86: Booting SMP configuration: [ 0.057027] .... node #0, CPUs: #1 #2 #3 [ 0.061235] smp: Brought up 1 node, 4 CPUs [ 0.063017] smpboot: Max logical packages: 1 [ 0.064021] smpboot: Total of 4 processors activated (19199.96 BogoMIPS) [ 0.161781] node 0 deferred pages initialised in 94ms [ 0.164106] devtmpfs: initialized [ 0.166337] x86/mm: Memory block size: 128MB [ 0.172315] gcov: version magic: 0x41383552 [ 0.176376] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.177108] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.178609] pinctrl core: initialized pinctrl subsystem [ 0.179212] [ 0.180009] ************************************************************* [ 0.181018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.182017] ** ** [ 0.183014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.184020] ** ** [ 0.185013] ** This means that this kernel is built to expose internal ** [ 0.186019] ** IOMMU data structures, which may compromise security on ** [ 0.187015] ** your system. ** [ 0.188017] ** ** [ 0.189020] ** If you see this message and you are not debugging the ** [ 0.190018] ** kernel, report this immediately to your vendor! ** [ 0.191016] ** ** [ 0.192019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.193020] ************************************************************* [ 0.195054] NET: Registered protocol family 16 [ 0.196979] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.197068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.198068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.200025] cpuidle: using governor menu [ 0.202253] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.205523] PCI: Using configuration type 1 for base access [ 0.207100] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.219178] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.223024] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.226080] cryptd: max_cpu_qlen set to 1000 [ 0.233095] ACPI: Added _OSI(Module Device) [ 0.234018] ACPI: Added _OSI(Processor Device) [ 0.235019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.236014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.240857] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.242739] ACPI: Interpreter enabled [ 0.243061] ACPI: PM: (supports S0 S3 S4 S5) [ 0.244013] ACPI: Using IOAPIC for interrupt routing [ 0.245098] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.246288] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.254299] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.255044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.256021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.257093] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.259477] acpiphp: Slot [2] registered [ 0.260107] acpiphp: Slot [5] registered [ 0.261096] acpiphp: Slot [6] registered [ 0.262273] acpiphp: Slot [7] registered [ 0.263105] acpiphp: Slot [8] registered [ 0.264096] acpiphp: Slot [9] registered [ 0.265096] acpiphp: Slot [10] registered [ 0.266107] acpiphp: Slot [3] registered [ 0.267080] acpiphp: Slot [4] registered [ 0.268086] acpiphp: Slot [11] registered [ 0.269123] acpiphp: Slot [12] registered [ 0.270286] acpiphp: Slot [13] registered [ 0.271084] acpiphp: Slot [14] registered [ 0.272075] acpiphp: Slot [15] registered [ 0.272871] acpiphp: Slot [16] registered [ 0.273073] acpiphp: Slot [17] registered [ 0.273928] acpiphp: Slot [18] registered [ 0.274059] acpiphp: Slot [19] registered [ 0.274992] acpiphp: Slot [20] registered [ 0.275050] acpiphp: Slot [21] registered [ 0.276051] acpiphp: Slot [22] registered [ 0.277248] acpiphp: Slot [23] registered [ 0.278275] acpiphp: Slot [24] registered [ 0.279082] acpiphp: Slot [25] registered [ 0.280056] acpiphp: Slot [26] registered [ 0.281061] acpiphp: Slot [27] registered [ 0.282091] acpiphp: Slot [28] registered [ 0.283105] acpiphp: Slot [29] registered [ 0.284073] acpiphp: Slot [30] registered [ 0.285072] acpiphp: Slot [31] registered [ 0.286007] PCI host bridge to bus 0000:00 [ 0.287012] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.288031] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.289015] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.290015] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.291016] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.292020] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.293142] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.294676] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.297244] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.307015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.313064] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.315013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.318016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.320015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.329224] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.334135] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.338059] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.340782] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.347016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.362014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.368779] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.375485] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.382014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.388026] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.404018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.413000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.422017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.429018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.451015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.466282] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.481027] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.495015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.517016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.532631] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.551016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.562017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.599019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.618225] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.630016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.638016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.658020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.670230] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.682016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.690015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.706022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.719269] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.722392] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.725623] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.728589] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.732481] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.738251] iommu: Default domain type: Passthrough [ 0.740415] SCSI subsystem initialized [ 0.742105] ACPI: bus type USB registered [ 0.744121] usbcore: registered new interface driver usbfs [ 0.746051] usbcore: registered new interface driver hub [ 0.748059] usbcore: registered new device driver usb [ 0.750152] pps_core: LinuxPPS API ver. 1 registered [ 0.752009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.755051] PTP clock support registered [ 0.758074] EDAC MC: Ver: 3.0.0 [ 0.760134] PCI: Using ACPI for IRQ routing [ 0.762928] NetLabel: Initializing [ 0.765010] NetLabel: domain hash size = 128 [ 0.768012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.770101] NetLabel: unlabeled traffic allowed by default [ 0.773069] vgaarb: loaded [ 0.775258] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.778018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.789000] clocksource: Switched to clocksource kvm-clock [ 0.947663] VFS: Disk quotas dquot_6.6.0 [ 0.949700] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.954028] *** VALIDATE ramfs *** [ 0.955693] *** VALIDATE hugetlbfs *** [ 0.958558] pnp: PnP ACPI init [ 0.962974] pnp: PnP ACPI: found 6 devices [ 0.985789] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.990908] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.993641] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.996894] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.000171] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.004129] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.008366] NET: Registered protocol family 2 [ 1.011683] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.017910] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.022818] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.030659] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.035971] TCP: Hash tables configured (established 65536 bind 65536) [ 1.040320] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.044986] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.048744] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.053187] NET: Registered protocol family 1 [ 1.056784] RPC: Registered named UNIX socket transport module. [ 1.059498] RPC: Registered udp transport module. [ 1.061689] RPC: Registered tcp transport module. [ 1.064247] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.067504] NET: Registered protocol family 44 [ 1.068730] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.071067] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.074104] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.077507] PCI: CLS 0 bytes, default 64 [ 1.079953] Unpacking initramfs... [ 2.901992] debug: unmapping init [mem 0xffff90a93cc54000-0xffff90a93ffbffff] [ 2.907587] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.910256] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.914110] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229833f6470, max_idle_ns: 440795327230 ns [ 3.577257] Initialise system trusted keyrings [ 3.580723] Key type blacklist registered [ 3.583293] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.594229] zbud: loaded [ 3.597232] *** VALIDATE nfs *** [ 3.598824] *** VALIDATE nfs4 *** [ 3.600822] pstore: using deflate compression [ 3.605673] Platform Keyring initialized [ 3.755416] NET: Registered protocol family 38 [ 3.757443] Key type asymmetric registered [ 3.760146] Asymmetric key parser 'x509' registered [ 3.762520] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.766879] io scheduler mq-deadline registered [ 3.768354] io scheduler kyber registered [ 3.769693] io scheduler bfq registered [ 3.771228] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.774415] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.777690] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.780685] ACPI: Power Button [PWRF] [ 3.788309] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.797288] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.812286] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.820689] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.839059] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.869567] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.899889] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.906247] Non-volatile memory driver v1.3 [ 3.908447] Linux agpgart interface v0.103 [ 3.939712] virtio_blk virtio1: [vda] 134592 512-byte logical blocks (68.9 MB/65.7 MiB) [ 3.942500] vda: detected capacity change from 0 to 68911104 [ 3.959834] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.963176] vdb: detected capacity change from 0 to 1073741824 [ 3.980944] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.984142] vdc: detected capacity change from 0 to 2621440000 [ 4.007231] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.011573] vdd: detected capacity change from 0 to 2621440000 [ 4.034923] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.038512] vde: detected capacity change from 0 to 4294967296 [ 4.062067] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.067171] vdf: detected capacity change from 0 to 4294967296 [ 4.074596] libphy: Fixed MDIO Bus: probed [ 4.083769] usbcore: registered new interface driver usbserial_generic [ 4.087827] usbserial: USB Serial support registered for generic [ 4.091368] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.097846] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.101084] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.104043] mousedev: PS/2 mouse device common for all mice [ 4.108540] rtc_cmos 00:05: RTC can wake from S4 [ 4.111574] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.115578] rtc_cmos 00:05: registered as rtc0 [ 4.118283] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.122316] intel_pstate: CPU model not supported [ 4.123100] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.132157] hid: raw HID events driver (C) Jiri Kosina [ 4.134361] usbcore: registered new interface driver usbhid [ 4.134493] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.136265] usbhid: USB HID core driver [ 4.136444] drop_monitor: Initializing network drop monitor service [ 4.143013] Initializing XFRM netlink socket [ 4.144759] NET: Registered protocol family 10 [ 4.147751] Segment Routing with IPv6 [ 4.149333] NET: Registered protocol family 17 [ 4.151322] mpls_gso: MPLS GSO support [ 4.158962] RAS: Correctable Errors collector initialized. [ 4.163059] AVX version of gcm_enc/dec engaged. [ 4.166075] AES CTR mode by8 optimization enabled [ 4.255228] sched_clock: Marking stable (4255199884, 0)->(5379195187, -1123995303) [ 4.259539] registered taskstats version 1 [ 4.262431] Loading compiled-in X.509 certificates [ 4.264755] zswap: loaded using pool lzo/zbud [ 4.286199] Key type big_key registered [ 4.297459] Key type encrypted registered [ 4.299065] ima: No TPM chip found, activating TPM-bypass! [ 4.300397] ima: Allocated hash algorithm: sha1 [ 4.302242] ima: No architecture policies found [ 4.303474] evm: Initialising EVM extended attributes: [ 4.305719] evm: security.selinux [ 4.307395] evm: security.ima [ 4.308405] evm: security.capability [ 4.309882] evm: HMAC attrs: 0x1 [ 4.312116] rtc_cmos 00:05: setting system clock to 2026-03-15 20:37:54 UTC (1773607074) [ 4.318312] debug: unmapping init [mem 0xffffffffb4a03000-0xffffffffb4bfffff] [ 4.321824] debug: unmapping init [mem 0xffffffffb3782000-0xffffffffb3a58fff] [ 4.331091] Write protecting the kernel read-only data: 28672k [ 4.334607] debug: unmapping init [mem 0xffffffffb1e03000-0xffffffffb1ffffff] [ 4.336676] debug: unmapping init [mem 0xffffffffb2714000-0xffffffffb27fffff] [ 4.373601] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.383547] systemd[1]: Detected virtualization kvm. [ 4.385795] systemd[1]: Detected architecture x86-64. [ 4.387381] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.412647] systemd[1]: No hostname configured. [ 4.415304] systemd[1]: Set hostname to . [ 4.417709] random: systemd: uninitialized urandom read (16 bytes read) [ 4.421218] systemd[1]: Initializing machine ID from random generator. [ 4.464797] random: ln: uninitialized urandom read (6 bytes read) [ 4.597671] random: systemd: uninitialized urandom read (16 bytes read) [ 4.599863] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.603659] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.609227] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.458240] device-mapper: uevent: version 1.0.3 [ 5.460477] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 6.411513] random: fast init done [ 6.425400] virtio_net virtio0 ens2: renamed from eth0 [ 6.504097] scsi host0: ata_piix [ 6.534648] scsi host1: ata_piix [ 6.537078] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.540111] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.814993] random: crng init done [ 10.817205] random: 7 urandom warning(s) missed due to ratelimiting [ 11.000717] dracut-initqueue[589]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 11.818098] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.334902] printk: systemd: 24 output lines suppressed due to ratelimiting [ 13.681331] SELinux: Disabled at runtime. [ 13.750749] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.757376] systemd[1]: Detected virtualization kvm. [ 13.759333] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 14.418647] systemd[1]: initrd-switch-root.service: Succeeded. [ 14.422545] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 14.431222] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 14.441667] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 14.446464] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 14.458070] systemd[1]: Starting Journal Service... Starting Journal Service... [ 14.472333] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. [ OK ] Reached target RPC Port Mapper. Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on Process Core Dump Socket. [ 14.645812] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ 15.141290] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.723208] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.777445] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 15.935918] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 15.961848] EDAC sbridge: Ver: 1.1.2 [ 18.154970] Key type dns_resolver registered [ 18.529781] NFS: Registering the id_resolver key type [ 18.533216] Key type id_resolver registered [ 18.536205] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg125-server login: [ 33.233556] hrtimer: interrupt took 2811856 ns [ 94.277716] libcfs: loading out-of-tree module taints kernel. [ 94.397445] Key type ._llcrypt registered [ 94.404282] Key type .llcrypt registered [ 94.628990] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 125.846694] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 129.740051] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 129.794831] alg: No test for adler32 (adler32-zlib) [ 132.806916] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 134.640678] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 136.645108] Key type lgssc registered [ 140.273128] Lustre: Echo OBD driver; http://www.lustre.org/ [ 170.134436] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 246.026627] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 270.019840] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 270.186275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 271.717626] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 271.798798] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 271.993730] Lustre: lustre-MDT0000: new disk, initializing [ 272.276854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 272.342229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 279.744838] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 303.000075] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 303.205630] Lustre: 6548:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 303.332445] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 303.339404] Lustre: Skipped 1 previous similar message [ 303.651996] Lustre: lustre-MDT0001: new disk, initializing [ 303.740353] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 303.775422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 303.792194] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 311.023159] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 317.202038] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 331.655366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 331.977781] Lustre: lustre-OST0000: new disk, initializing [ 331.982699] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 332.071021] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 337.958416] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 337.981289] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 338.115902] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 341.462471] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 362.238669] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 362.563685] Lustre: lustre-OST0001: new disk, initializing [ 362.565862] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 362.698256] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 370.286730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 370.309116] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 370.486790] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 373.002171] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 389.331827] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 400.689915] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 414.781287] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing check_logdir /tmp/testlogs/ [ 423.047180] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing yml_node [ 429.804750] Lustre: DEBUG MARKER: Client: 2.17.51.2 [ 433.358791] Lustre: DEBUG MARKER: MDS: 2.17.51.2 [ 437.076683] Lustre: DEBUG MARKER: OSS: 2.17.51.2 [ 439.445627] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Mar 15 16:45:06 EDT 2026 [ 464.969864] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 467.040786] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 488.418800] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 488.425504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 488.436059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 488.449556] Lustre: Skipped 3 previous similar messages [ 488.484652] Lustre: Skipped 3 previous similar messages [ 494.012371] Lustre: server umount lustre-MDT0000 complete [ 498.674870] LustreError: 6555:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 498.705840] LustreError: 6555:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 8 previous similar messages [ 499.792429] LustreError: 6540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773607569 with bad export cookie 3356160497137070737 [ 499.800072] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 499.819585] LustreError: 6540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 503.780174] LustreError: 9700:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 503.781747] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 503.827198] LustreError: 9700:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 503.871883] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 503.883913] Lustre: Skipped 1 previous similar message [ 504.802309] LustreError: 6556:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 504.844597] LustreError: 6556:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 506.400448] Lustre: server umount lustre-MDT0001 complete [ 511.737208] Lustre: server umount lustre-OST0000 complete [ 517.090547] Lustre: server umount lustre-OST0001 complete [ 528.231334] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 547.349384] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 600.029539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 600.448769] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 600.500368] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 600.639396] Lustre: lustre-MDT0000: new disk, initializing [ 600.869541] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 600.920894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 606.944862] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 621.761064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 622.442074] Lustre: 15068:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 622.623369] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 622.629752] Lustre: Skipped 1 previous similar message [ 622.840888] Lustre: lustre-MDT0001: new disk, initializing [ 622.999982] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 623.124352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 623.143733] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 630.715966] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 642.752980] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 648.732869] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 659.127295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 659.589405] Lustre: lustre-OST0000: new disk, initializing [ 659.593321] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 659.704575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 661.296782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 661.314908] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 661.450916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 668.224989] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 678.521954] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 684.454751] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 684.692917] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 690.968468] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 691.221996] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 697.326901] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 697.330449] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 697.348673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 697.348682] Lustre: Skipped 2 previous similar messages [ 697.426514] Lustre: Skipped 2 previous similar messages [ 699.827697] Lustre: server umount lustre-OST0000 complete [ 707.558935] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 707.573110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 707.579842] Lustre: Skipped 1 previous similar message [ 712.673518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 712.693470] Lustre: Skipped 1 previous similar message [ 720.355500] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 720.955750] Lustre: server umount lustre-MDT0000 complete [ 722.918133] LustreError: 15079:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 722.962881] LustreError: 15079:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 727.720752] LustreError: 15061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773607797 with bad export cookie 3356160497137072270 [ 727.737286] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 727.750630] LustreError: 15061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 728.306969] Lustre: server umount lustre-MDT0001 complete [ 737.465378] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:50:03 (1773607803) [ 749.496804] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 750.190980] LustreError: 18935:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 750.349278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 758.480246] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 760.801564] LustreError: 18936:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 760.847369] LustreError: 18936:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 771.180213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 779.540785] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 788.006135] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 794.567263] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 806.917123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 820.843693] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 830.091649] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 835.546601] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 847.331817] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 847.339585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 847.376853] Lustre: Skipped 3 previous similar messages [ 847.387357] Lustre: Skipped 3 previous similar messages [ 850.868668] Lustre: server umount lustre-MDT0000 complete [ 852.450561] LustreError: 18936:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 852.521988] LustreError: 18936:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [ 855.984761] LustreError: 18918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773607926 with bad export cookie 3356160497137073145 [ 855.996805] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 855.997018] LustreError: 18918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 856.473949] Lustre: server umount lustre-MDT0001 complete [ 872.613301] Lustre: server umount lustre-OST0000 complete [ 906.145540] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 921.531855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 922.445573] LustreError: 22994:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 922.572071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 922.581375] Lustre: Skipped 2 previous similar messages [ 930.129481] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 942.289454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 950.466496] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 955.202705] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 964.809737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 975.276716] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 988.735968] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 989.085330] Lustre: lustre-OST0001: new disk, initializing [ 989.090693] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 990.822565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 990.841200] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 991.095510] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1000.514196] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1014.647435] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1020.049689] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 1042.415729] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1042.427449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1042.446171] Lustre: Skipped 2 previous similar messages [ 1042.461387] Lustre: Skipped 3 previous similar messages [ 1048.051958] Lustre: server umount lustre-MDT0000 complete [ 1052.643701] LustreError: 22994:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1052.730601] LustreError: 22994:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 6 previous similar messages [ 1053.126256] LustreError: 22976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773608123 with bad export cookie 3356160497137073999 [ 1053.131201] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1053.149437] LustreError: 22976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1053.637926] Lustre: server umount lustre-MDT0001 complete [ 1070.801736] Lustre: server umount lustre-OST0000 complete [ 1073.654624] Lustre: 3670:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1773608127/real 1773608127] req@ffff90a9b76f4000 x1859761947829888/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1773608143 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1073.730605] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1073.755645] Lustre: Skipped 1 previous similar message [ 1076.228433] Lustre: server umount lustre-OST0001 complete [ 1101.899872] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 1115.552612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1116.324587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1116.334095] Lustre: Skipped 3 previous similar messages [ 1122.743611] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1134.134730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1140.416426] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1144.813469] Lustre: 29370:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1153.685977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1163.784872] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1176.733886] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1185.304837] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1195.952799] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1200.984603] Lustre: 31229:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1216.481512] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1216.492696] LustreError: Skipped 1 previous similar message [ 1216.511807] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1216.564665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1216.577190] Lustre: Skipped 4 previous similar messages [ 1218.908603] Lustre: server umount lustre-MDT0000 complete [ 1223.144946] LustreError: 28252:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1223.205652] LustreError: 28252:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 14 previous similar messages [ 1225.369672] LustreError: 29754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773608295 with bad export cookie 3356160497137075287 [ 1225.407363] LustreError: 29754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1225.415749] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1225.915575] Lustre: server umount lustre-MDT0001 complete [ 1242.737961] Lustre: server umount lustre-OST0000 complete [ 1244.514621] Lustre: 3672:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1773608298/real 1773608298] req@ffff90a88358d880 x1859761947890944/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1773608314 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1244.572883] Lustre: 3672:0:(client.c:2478:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1246.431840] Lustre: 3670:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1773608300/real 1773608300] req@ffff90a9b49d7b80 x1859761947891200/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1773608316 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1248.918816] Lustre: server umount lustre-OST0001 complete [ 1261.479365] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:58:47 (1773608327) [ 1305.789222] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 1322.009720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1330.101737] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1344.067990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1351.206912] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1357.347594] Lustre: 35155:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1367.408249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1377.203436] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1389.086765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1389.556875] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1389.562339] Lustre: Skipped 6 previous similar messages [ 1400.191985] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1409.728256] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1414.920537] Lustre: 37022:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1430.505527] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1430.518076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1430.538596] Lustre: Skipped 6 previous similar messages [ 1430.553750] Lustre: Skipped 7 previous similar messages [ 1434.742556] Lustre: server umount lustre-MDT0000 complete [ 1440.621710] LustreError: 34019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773608510 with bad export cookie 3356160497137076400 [ 1440.643193] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1440.660650] LustreError: 34019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1441.377766] Lustre: server umount lustre-MDT0001 complete [ 1448.107202] Lustre: server umount lustre-OST0000 complete [ 1454.626448] Lustre: server umount lustre-OST0001 complete [ 1468.052936] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 17:02:14 (1773608534) [ 1493.102680] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 1508.474055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1509.078278] LustreError: 39171:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1509.131907] LustreError: 39171:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 21 previous similar messages [ 1515.699561] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1529.131127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1537.402732] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1541.501091] Lustre: 40287:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1551.206584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1562.426565] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1577.045331] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1587.784089] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1598.502980] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1604.268263] Lustre: 42141:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1610.783397] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1610.793747] Lustre: Skipped 1 previous similar message [ 1616.866601] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1616.879285] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1616.906079] Lustre: Skipped 3 previous similar messages [ 1616.915776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1616.922979] Lustre: Skipped 2 previous similar messages [ 1621.220963] Lustre: server umount lustre-MDT0000 complete [ 1626.358891] LustreError: 40670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773608696 with bad export cookie 3356160497137077506 [ 1626.363628] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1626.381397] LustreError: 40670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1632.970070] Lustre: server umount lustre-MDT0001 complete [ 1644.051297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1651.684127] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x2e937a30c10dd1a8 [ 1651.703572] Lustre: MGC192.168.201.125@tcp: Connection restored to 0@lo (at 0@lo) [ 1659.641736] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1661.623392] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1673.691836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1679.890601] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1679.903481] Lustre: Skipped 1 previous similar message [ 1681.377529] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1689.359654] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1691.571205] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1697.187201] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1699.568644] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1712.026846] Lustre: server umount lustre-MDT0000 complete [ 1717.620408] LustreError: 42143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773608787 with bad export cookie 3356160497137078696 [ 1717.628347] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1717.651415] LustreError: 42143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1718.125620] Lustre: server umount lustre-MDT0001 complete [ 1734.425459] Lustre: server umount lustre-OST0000 complete [ 1736.223538] Lustre: 3672:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1773608790/real 1773608790] req@ffff90a9b491ca80 x1859761948064384/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1773608806 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1740.267784] Lustre: server umount lustre-OST0001 complete [ 1754.917864] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 17:07:01 (1773608821) [ 1778.597504] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 1793.515422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1799.503943] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1812.969066] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1822.393846] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1827.199554] Lustre: 47664:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1837.280947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1847.473374] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1859.246041] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1869.359517] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 1879.050551] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1890.844919] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1915.875346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1915.884904] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1915.893729] Lustre: Skipped 10 previous similar messages [ 1915.897319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1915.901923] Lustre: Skipped 19 previous similar messages [ 1916.536752] Lustre: server umount lustre-MDT0000 complete [ 1921.916034] LustreError: 46523:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773608992 with bad export cookie 3356160497137079494 [ 1921.943833] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1921.954059] LustreError: 46523:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1922.436375] Lustre: server umount lustre-MDT0001 complete [ 1938.690709] Lustre: server umount lustre-OST0000 complete [ 1941.989063] Lustre: 3670:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1773608996/real 1773608996] req@ffff90a884af3b80 x1859761948139904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1773609012 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1942.038831] Lustre: 3670:0:(client.c:2478:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1944.754520] Lustre: server umount lustre-OST0001 complete [ 1959.423507] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 17:10:26 (1773609026) [ 1963.048577] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1966.281910] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 17:10:32 (1773609032) [ 1980.700583] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 1994.599683] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 2029.039160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2029.532704] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2029.550902] Lustre: Skipped 1 previous similar message [ 2029.594175] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2029.831847] Lustre: lustre-MDT0000: new disk, initializing [ 2030.115387] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2030.132132] Lustre: Skipped 10 previous similar messages [ 2030.162350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2037.457210] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2049.007745] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2060.253178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2060.679940] Lustre: lustre-OST0000: new disk, initializing [ 2060.694775] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2060.706936] Lustre: Skipped 1 previous similar message [ 2062.719076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2062.736308] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2062.829160] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2069.987499] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2080.554506] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2084.117783] Lustre: DEBUG MARKER: create test files [ 2138.446652] Lustre: server umount lustre-OST0000 complete [ 2153.220943] Lustre: server umount lustre-MDT0000 complete [ 2174.711892] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 2179.768740] Key type lgssc unregistered [ 2180.297484] LNet: 55139:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2180.315780] LNetError: 55139:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2181.351706] LNet: Removed LNI 192.168.201.125@tcp [ 2182.872173] Key type .llcrypt unregistered [ 2182.876951] Key type ._llcrypt unregistered [ 2231.374874] Key type ._llcrypt registered [ 2231.379930] Key type .llcrypt registered [ 2231.662407] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 2234.448394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2234.506678] alg: No test for adler32 (adler32-zlib) [ 2235.733515] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 2236.078835] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 2237.914773] Key type lgssc registered [ 2240.285756] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2252.863820] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2252.899613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2254.784665] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.821479] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2271.587799] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2281.919894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2282.459926] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2292.368684] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2299.855271] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2300.079513] Lustre: 57747:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 2300.090918] Lustre: 57747:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 2300.107775] Lustre: 57747:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2300.114265] Lustre: 57747:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 2300.122679] Lustre: 57747:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2300.131656] Lustre: 57747:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2300.693724] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2307.501951] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2315.909052] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2330.601777] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2330.612124] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2330.675234] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2335.719077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2340.835940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2341.343144] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2341.830487] Lustre: server umount lustre-OST0000 complete [ 2348.234287] Lustre: server umount lustre-MDT0000 complete [ 2372.209743] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 2377.491586] Key type lgssc unregistered [ 2378.093307] LNet: 59254:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2378.114663] LNetError: 59254:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2378.203310] LNet: Removed LNI 192.168.201.125@tcp [ 2379.203482] Key type .llcrypt unregistered [ 2379.205775] Key type ._llcrypt unregistered [ 2429.553133] Key type ._llcrypt registered [ 2429.555805] Key type .llcrypt registered [ 2429.884169] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 2432.669841] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2432.706019] alg: No test for adler32 (adler32-zlib) [ 2433.911382] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 2434.219779] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 2435.903338] Key type lgssc registered [ 2437.463593] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2451.193871] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2451.222631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2453.540334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2460.895893] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2470.210576] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2481.686252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2482.759234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2493.281266] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2498.109218] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2498.278360] Lustre: 61964:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2498.295461] Lustre: 61964:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2498.304588] Lustre: 61964:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2498.310922] Lustre: 61964:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 2498.319621] Lustre: 61964:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2498.325418] Lustre: 61964:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2502.124900] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2509.354064] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2523.615831] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2523.622674] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2523.631539] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2525.953843] Lustre: server umount lustre-OST0000 complete [ 2532.521487] Lustre: server umount lustre-MDT0000 complete [ 2551.352889] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 2556.492928] Key type lgssc unregistered [ 2557.161808] LNet: 63427:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2557.178487] LNetError: 63427:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2557.267290] LNet: Removed LNI 192.168.201.125@tcp [ 2558.717374] Key type .llcrypt unregistered [ 2558.719860] Key type ._llcrypt unregistered [ 2590.222756] Key type ._llcrypt registered [ 2590.228344] Key type .llcrypt registered [ 2590.377928] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 2613.115902] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 2614.846795] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2615.028146] alg: No test for adler32 (adler32-zlib) [ 2616.842617] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 2617.248430] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 2619.023202] Key type lgssc registered [ 2621.142863] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2670.047566] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2670.090576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2671.520656] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2671.570912] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2671.728203] Lustre: lustre-MDT0000: new disk, initializing [ 2671.981800] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2672.034670] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2678.359092] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2694.189968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2694.540411] Lustre: 66551:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2694.710541] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2694.734497] Lustre: Skipped 1 previous similar message [ 2694.942371] Lustre: lustre-MDT0001: new disk, initializing [ 2695.148290] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2695.218426] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2695.238390] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2702.257434] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2712.942050] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2718.507693] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2728.659708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2729.030549] Lustre: lustre-OST0000: new disk, initializing [ 2729.041843] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2729.159139] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2735.810413] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2735.844219] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2736.057543] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2738.527207] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2749.430302] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2755.911584] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2756.407659] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2762.778214] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2763.182119] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2766.824443] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2766.825217] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2766.828477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2766.858412] Lustre: Skipped 1 previous similar message [ 2766.899207] LustreError: Skipped 1 previous similar message [ 2771.943062] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2771.958339] Lustre: Skipped 2 previous similar messages [ 2772.401213] Lustre: server umount lustre-OST0000 complete [ 2782.193641] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2782.195172] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2782.211032] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2784.472571] Lustre: server umount lustre-MDT0000 complete [ 2790.037115] LustreError: 66544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773609860 with bad export cookie 15581241238916337379 [ 2790.045646] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2790.054987] LustreError: 66544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2790.681848] Lustre: server umount lustre-MDT0001 complete [ 2803.251740] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 17:24:29 (1773609869) [ 2819.961507] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 2880.490786] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 2896.419234] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 2950.913752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2951.218979] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2951.254618] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2951.348244] Lustre: lustre-MDT0000: new disk, initializing [ 2951.481414] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2951.518981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2957.539315] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2971.833874] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2972.114302] Lustre: 74160:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2972.235813] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2972.239978] Lustre: Skipped 1 previous similar message [ 2972.392953] Lustre: lustre-MDT0001: new disk, initializing [ 2972.553718] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2972.640054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2972.659944] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2979.090054] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 2989.816488] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2995.095869] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3005.938339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3006.597147] Lustre: lustre-OST0000: new disk, initializing [ 3006.607087] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3006.853916] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3015.301890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3015.327017] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3015.486380] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3017.092184] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3027.128484] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3034.999412] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3035.355479] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3041.122834] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3041.473539] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3046.379065] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3046.383158] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3046.400732] Lustre: Skipped 1 previous similar message [ 3046.406166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3046.421959] Lustre: Skipped 2 previous similar messages [ 3050.487471] Lustre: server umount lustre-OST0000 complete [ 3059.687951] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3059.704090] LustreError: Skipped 1 previous similar message [ 3059.717444] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3059.736713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3063.951545] Lustre: server umount lustre-MDT0000 complete [ 3065.890862] LustreError: 76461:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3065.921962] LustreError: 76461:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [ 3069.987456] LustreError: 74152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773610140 with bad export cookie 15581241238916338261 [ 3070.001285] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3070.020592] LustreError: 74152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3070.863115] Lustre: server umount lustre-MDT0001 complete [ 3083.982229] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 17:29:10 (1773610150) [ 3093.779688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3095.037183] LustreError: 78172:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3095.235094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3100.664168] LustreError: 78173:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3101.950555] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3105.776488] LustreError: 78172:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3110.902188] LustreError: 78173:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3114.661320] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3120.422812] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3129.855757] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3136.296407] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3145.555373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3146.190179] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3146.202938] Lustre: Skipped 1 previous similar message [ 3154.956933] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3163.469371] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3169.380328] Lustre: 80510:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting ed705429-5479-4af9-900f-185a469c5761 at adminstrative request [ 3170.796324] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3170.824930] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 3181.036969] Lustre: MGS: Client ed705429-5479-4af9-900f-185a469c5761 (at 0@lo) reconnecting [ 3181.050393] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd83bb05a954695c7 to 0xd83bb05a95469867 [ 3181.070252] Lustre: MGC192.168.201.125@tcp: Connection restored to 0@lo (at 0@lo) [ 3210.306254] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3210.593247] Lustre: lustre-OST0001: new disk, initializing [ 3210.599818] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3210.719230] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3219.566711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3219.593497] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3219.694700] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3220.016874] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3231.277072] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3240.428156] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3240.433190] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3240.456237] LustreError: Skipped 1 previous similar message [ 3240.458485] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3240.479537] Lustre: Skipped 2 previous similar messages [ 3240.520320] Lustre: Skipped 3 previous similar messages [ 3241.654596] Lustre: server umount lustre-OST0001 complete [ 3245.551779] LustreError: 79768:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3245.600643] LustreError: 79768:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 3253.636290] Lustre: server umount lustre-OST0000 complete [ 3260.901590] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3260.905272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3260.950655] Lustre: Skipped 3 previous similar messages [ 3260.962698] Lustre: Skipped 4 previous similar messages [ 3265.707115] Lustre: server umount lustre-MDT0000 complete [ 3266.033425] LustreError: 78885:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3266.063831] LustreError: 78885:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 3271.561349] LustreError: 78153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773610341 with bad export cookie 15581241238916339815 [ 3271.584339] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3272.287640] Lustre: server umount lustre-MDT0001 complete [ 3284.850813] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 17:32:31 (1773610351) [ 3287.269106] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3289.569563] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 17:32:36 (1773610356) [ 3291.794621] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3294.041914] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 17:32:40 (1773610360) [ 3296.108759] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3298.667848] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 17:32:45 (1773610365) [ 3301.004529] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3303.535782] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 17:32:50 (1773610370) [ 3305.891922] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3313.870100] loop: module loaded [ 3313.902066] loop0: detected capacity change from 0 to 4294967296 [ 3313.979089] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3316.691584] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3321.290143] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3327.281402] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3361.310184] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3361.400254] LustreError: 84910:0:(obd_config.c:880:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3361.405963] LustreError: 84910:0:(obd_mount.c:217:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3361.412334] LustreError: 84910:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3361.423963] LustreError: 84910:0:(super25.c:186:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3387.041099] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 3405.322212] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 3459.519470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3459.950119] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3460.026971] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3460.147948] Lustre: lustre-MDT0000: new disk, initializing [ 3460.279820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3460.301974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3466.633967] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3480.120994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3480.342191] Lustre: 87775:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3480.769523] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3486.585262] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3495.424605] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3500.792608] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3511.384461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3517.620181] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3517.832841] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3522.301519] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3533.548158] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3539.717712] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3540.027278] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3546.523362] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3546.932295] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3553.776831] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3553.785825] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3553.805803] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3553.805814] Lustre: Skipped 1 previous similar message [ 3553.807886] LustreError: Skipped 3 previous similar messages [ 3553.879559] Lustre: Skipped 1 previous similar message [ 3555.998326] Lustre: server umount lustre-OST0000 complete [ 3567.902569] Lustre: server umount lustre-MDT0000 complete [ 3569.125185] LustreError: 87786:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3569.144935] LustreError: 87786:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 3573.341513] LustreError: 89517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773610643 with bad export cookie 15581241238916340431 [ 3573.346260] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3573.352311] LustreError: 89517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3573.986727] Lustre: server umount lustre-MDT0001 complete [ 3586.184958] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 17:37:33 (1773610653) [ 3594.240830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3601.265639] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3613.630759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3620.337797] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3628.097764] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3633.396918] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3642.015825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3650.519207] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3658.051891] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3665.214565] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3666.922785] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 3673.253643] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3673.577700] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3682.909016] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3683.162900] Lustre: lustre-OST0001: new disk, initializing [ 3683.168069] Lustre: Skipped 2 previous similar messages [ 3683.182544] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3683.200557] Lustre: Skipped 3 previous similar messages [ 3692.768268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3692.785429] Lustre: Skipped 2 previous similar messages [ 3692.812292] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3692.823085] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3693.005800] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3702.926692] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3709.139690] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3709.489719] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3715.729882] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3716.126874] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3739.425832] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3776.980548] Lustre: Failing over lustre-MDT0000 [ 3777.623411] Lustre: server umount lustre-MDT0000 complete [ 3778.024284] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3778.043192] LustreError: Skipped 1 previous similar message [ 3778.058186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3778.084364] Lustre: Skipped 2 previous similar messages [ 3778.103020] LustreError: 91787:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3778.127949] LustreError: 91787:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [ 3796.450361] Lustre: 64773:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1773610850/real 1773610850] req@000000006fb29fcb x1859764551459072/t0(0) o400->MGC192.168.201.125@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1773610866 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3796.510335] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3810.251261] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3810.255031] LDISKFS-fs (dm-0): recovery complete [ 3810.268813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3822.136358] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xd83bb05a954762db [ 3822.183895] Lustre: MGC192.168.201.125@tcp: Connection restored to 0@lo (at 0@lo) [ 3822.730740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3822.748027] Lustre: Skipped 6 previous similar messages [ 3822.847726] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3827.289077] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3827.289483] Lustre: 91783:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3828.205938] Lustre: 92499:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3828.228429] Lustre: 92499:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 3828.269043] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3828.409564] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3828.745108] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3829.072042] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3829.082229] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 1 previous similar message [ 3829.419592] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3829.436210] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 1 previous similar message [ 3829.451922] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3829.470224] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 3830.148928] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3830.170624] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 2 previous similar messages [ 3830.509933] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3830.523822] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 2 previous similar messages [ 3830.704085] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 3831.552474] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3831.578377] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 5 previous similar messages [ 3832.265460] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3832.289110] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 5 previous similar messages [ 3832.639803] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3832.651022] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 5 previous similar messages [ 3835.630535] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3835.661823] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 11 previous similar messages [ 3836.338845] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3836.369953] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 11 previous similar messages [ 3836.703628] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3836.717337] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 11 previous similar messages [ 3843.881200] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3843.917850] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 24 previous similar messages [ 3844.619116] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3844.631430] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 24 previous similar messages [ 3844.967338] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3844.984687] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 24 previous similar messages [ 3860.162000] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3860.210288] Lustre: 97509:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 49 previous similar messages [ 3860.861495] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3860.881214] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 49 previous similar messages [ 3861.209433] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3861.235367] LustreError: 97509:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 49 previous similar messages [ 3887.077096] Lustre: 97509:0:(ldlm_lib.c:2376:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3887.103414] Lustre: 97509:0:(ldlm_lib.c:2386:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3887.120017] Lustre: 97509:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 6225e5b3-01f9-4647-a8f8-1eb74c808b5c@192.168.201.25@tcp [ 3887.151361] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3887.167858] Lustre: 97509:0:(ldlm_lib.c:1896:abort_req_replay_queue()) @@@ aborted: req@000000004e49bb0a x1859764526061312/t0(8589934776) o35->6225e5b3-01f9-4647-a8f8-1eb74c808b5c@192.168.201.25@tcp:452/0 lens 392/0 e 0 to 0 dl 1773610967 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3887.234601] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3887.260552] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3887.273724] Lustre: lustre-MDT0000: Denying connection for new client 6225e5b3-01f9-4647-a8f8-1eb74c808b5c (at 192.168.201.25@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3887.351045] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3887.372493] Lustre: Skipped 2 previous similar messages [ 3887.390194] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3887.463624] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3887.489095] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3908.584582] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3908.605869] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3908.630362] Lustre: Skipped 3 previous similar messages [ 3908.641651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3908.650081] Lustre: Skipped 4 previous similar messages [ 3912.575172] Lustre: server umount lustre-OST0000 complete [ 3915.257219] LustreError: 98089:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3915.280381] LustreError: 98089:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 65 previous similar messages [ 3924.769624] Lustre: server umount lustre-OST0001 complete [ 3937.545434] Lustre: server umount lustre-MDT0000 complete [ 3942.976896] LustreError: 91767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773611013 with bad export cookie 15581241238916391643 [ 3942.980682] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3942.990658] LustreError: 91767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3943.589878] Lustre: server umount lustre-MDT0001 complete [ 3956.656707] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 17:43:43 (1773611023) [ 3966.913695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3966.919794] Lustre: *** cfs_fail_loc=197, val=0*** [ 3982.562591] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3987.680428] LustreError: 99474:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.125@tcp: failed processing log, type 4: rc = -110 [ 3992.879525] Lustre: *** cfs_fail_loc=197, val=0*** [ 3992.890265] Lustre: Skipped 65 previous similar messages [ 4022.996049] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4031.869881] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4035.494950] Lustre: server umount lustre-OST0000 complete [ 4047.744453] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 17:45:14 (1773611114) [ 4073.517187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4081.163891] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4091.242839] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4096.468523] LustreError: 101015:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 4096.488965] LustreError: 101015:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 4096.497853] LustreError: 101015:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 4096.840910] Lustre: server umount lustre-MDT0000 complete [ 4142.029634] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 4159.359631] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 4218.703970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4219.350679] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4219.399845] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4219.519926] Lustre: lustre-MDT0000: new disk, initializing [ 4219.745470] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4226.800871] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4241.567358] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4241.815827] Lustre: 105072:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4242.400151] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4248.718144] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4258.599510] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4263.751756] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4274.185120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4282.111449] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4283.862940] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4294.450571] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4301.639941] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4301.983909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4309.294876] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4309.655987] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4313.057165] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4313.073713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4313.079216] LustreError: Skipped 3 previous similar messages [ 4313.082483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4313.097623] Lustre: Skipped 6 previous similar messages [ 4313.104565] Lustre: Skipped 8 previous similar messages [ 4318.848447] Lustre: server umount lustre-OST0000 complete [ 4331.799895] Lustre: server umount lustre-MDT0000 complete [ 4333.538365] LustreError: 105083:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4333.559123] LustreError: 105083:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 11 previous similar messages [ 4337.708466] LustreError: 106816:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773611407 with bad export cookie 15581241238916402304 [ 4337.717413] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4337.725663] LustreError: 106816:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4338.611963] Lustre: server umount lustre-MDT0001 complete [ 4352.737102] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:50:18 (1773611418) [ 4371.362581] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 4377.776928] Key type lgssc unregistered [ 4378.501685] LNet: 109163:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4378.520984] LNetError: 109163:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4378.545595] LNet: Removed LNI 192.168.201.125@tcp [ 4380.561423] Key type .llcrypt unregistered [ 4380.568481] Key type ._llcrypt unregistered [ 4410.186745] Key type ._llcrypt registered [ 4410.189518] Key type .llcrypt registered [ 4410.347699] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 4433.347100] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 4435.133552] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4435.278358] alg: No test for adler32 (adler32-zlib) [ 4436.819082] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 4437.325511] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 4439.215326] Key type lgssc registered [ 4441.212823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4506.921644] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4506.966649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4508.511634] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4508.566494] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4508.780760] Lustre: lustre-MDT0000: new disk, initializing [ 4509.014631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4509.069406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4516.014618] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4531.683663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4532.387960] Lustre: lustre-OST0000: new disk, initializing [ 4532.403328] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4532.425797] Lustre: Skipped 1 previous similar message [ 4532.598679] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4539.079640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4539.095560] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4539.171000] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4541.752273] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4557.977046] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4558.278631] Lustre: lustre-OST0001: new disk, initializing [ 4558.294097] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4558.388646] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4568.342652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4568.370611] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4568.464323] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4569.624434] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4583.917461] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4583.927458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4583.946887] Lustre: Skipped 1 previous similar message [ 4583.968913] Lustre: Skipped 1 previous similar message [ 4589.024972] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4589.035173] Lustre: Skipped 1 previous similar message [ 4589.644976] Lustre: server umount lustre-MDT0000 complete [ 4599.795457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4607.135847] LustreError: 112497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773611677 with bad export cookie 10300330167293033237 [ 4607.156169] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4607.157866] LustreError: 112497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4607.337941] Lustre: server umount lustre-OST0000 complete [ 4614.348112] Lustre: server umount lustre-OST0001 complete [ 4627.036404] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 17:54:53 (1773611693) [ 4643.824060] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 4679.102965] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:55:45 (1773611745) [ 4738.829757] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 4755.585793] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 4804.277127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4804.777219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4804.865206] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4805.042266] Lustre: lustre-MDT0000: new disk, initializing [ 4805.205256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4805.236563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4811.877174] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4826.379607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4826.575556] Lustre: 120877:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4826.677719] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4826.684429] Lustre: Skipped 1 previous similar message [ 4826.833812] Lustre: lustre-MDT0001: new disk, initializing [ 4826.987055] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4827.062790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4827.087044] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4832.785929] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4844.065694] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4849.887318] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4859.986343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4860.405159] Lustre: lustre-OST0000: new disk, initializing [ 4860.409977] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4860.572248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4867.698995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4867.733119] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4867.998960] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4869.907454] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4880.711770] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4887.993681] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4888.267926] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4894.861364] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4895.135873] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4898.785711] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4898.791904] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4898.801249] LustreError: Skipped 1 previous similar message [ 4898.802405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4898.837949] Lustre: Skipped 1 previous similar message [ 4904.048402] Lustre: server umount lustre-OST0000 complete [ 4909.025404] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4909.080740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4909.093121] Lustre: Skipped 1 previous similar message [ 4914.153275] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4914.166554] Lustre: Skipped 1 previous similar message [ 4923.359673] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4923.737771] Lustre: server umount lustre-MDT0000 complete [ 4924.388419] LustreError: 120889:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4924.426724] LustreError: 120889:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [ 4928.954730] LustreError: 121658:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773611999 with bad export cookie 10300330167293034462 [ 4928.956855] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4928.967220] LustreError: 121658:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4929.356838] Lustre: server umount lustre-MDT0001 complete [ 4942.646224] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:00:09 (1773612009) [ 4951.743753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4952.474121] LustreError: 124889:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4952.525614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4957.685284] LustreError: 124890:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4958.032250] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4962.785928] LustreError: 124889:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4967.914499] LustreError: 124890:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4971.473855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4980.032326] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 4989.676828] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4996.581370] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5007.188197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5007.847241] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5007.859774] Lustre: Skipped 1 previous similar message [ 5016.841670] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5024.288782] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5029.445508] Lustre: *** cfs_fail_loc=159, val=0*** [ 5045.933361] Lustre: lustre-MDT0000: Client 9704fdb1-7732-402d-aeeb-de36a07f6fc4 (at 192.168.201.25@tcp) reconnecting [ 5047.821193] Lustre: *** cfs_fail_loc=159, val=0*** [ 5047.826471] Lustre: Skipped 5 previous similar messages [ 5063.287703] Lustre: lustre-MDT0000: Client 9704fdb1-7732-402d-aeeb-de36a07f6fc4 (at 192.168.201.25@tcp) reconnecting [ 5073.380364] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5073.394752] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5073.418051] Lustre: Skipped 1 previous similar message [ 5073.424581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5073.438686] Lustre: Skipped 2 previous similar messages [ 5074.420270] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5076.649372] Lustre: server umount lustre-OST0000 complete [ 5084.641981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5084.645922] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5084.674357] Lustre: Skipped 2 previous similar messages [ 5089.761369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5089.769419] Lustre: Skipped 5 previous similar messages [ 5095.391131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5095.650660] Lustre: server umount lustre-MDT0000 complete [ 5098.982138] LustreError: 125772:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5098.996424] LustreError: 125772:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5100.923067] LustreError: 126520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773612171 with bad export cookie 10300330167293035344 [ 5100.930471] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5100.943474] LustreError: 126520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5101.520358] Lustre: server umount lustre-MDT0001 complete [ 5117.703585] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 5122.052288] Key type lgssc unregistered [ 5122.495811] LNet: 128332:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5122.514156] LNetError: 128332:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5122.559070] LNet: Removed LNI 192.168.201.125@tcp [ 5123.948320] Key type .llcrypt unregistered [ 5123.954154] Key type ._llcrypt unregistered [ 5156.836697] Key type ._llcrypt registered [ 5156.842969] Key type .llcrypt registered [ 5157.049149] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 5158.605811] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5158.693884] alg: No test for adler32 (adler32-zlib) [ 5160.003685] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 5160.386308] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 5162.209606] Key type lgssc registered [ 5164.023723] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5173.498612] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5173.538041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5175.295480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5181.726496] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5185.521233] LustreError: 129732:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5185.558590] LustreError: 129732:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5190.624243] LustreError: 129733:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5193.063283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5193.435635] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5199.759130] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5206.770394] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5211.964895] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5220.835378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5221.262494] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5228.367564] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5235.662917] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5245.819324] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 5257.430674] Lustre: *** cfs_fail_loc=159, val=0*** [ 5273.732866] Lustre: lustre-MDT0000: Client a4ed2034-df08-45aa-a51a-47e96f06b288 (at 192.168.201.25@tcp) reconnecting [ 5277.074686] Lustre: *** cfs_fail_loc=159, val=0*** [ 5292.724198] Lustre: lustre-MDT0001: Client a4ed2034-df08-45aa-a51a-47e96f06b288 (at 192.168.201.25@tcp) reconnecting [ 5295.678303] Lustre: *** cfs_fail_loc=159, val=0*** [ 5295.697779] Lustre: Skipped 4 previous similar messages [ 5312.167811] Lustre: lustre-MDT0001: Client a4ed2034-df08-45aa-a51a-47e96f06b288 (at 192.168.201.25@tcp) reconnecting [ 5318.586951] Lustre: 132458:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5322.783914] Lustre: *** cfs_fail_loc=159, val=0*** [ 5322.790140] Lustre: Skipped 2 previous similar messages [ 5338.243735] Lustre: lustre-MDT0000: Client 627fc391-d4de-4da8-8bfe-06a854be75fe (at 192.168.201.25@tcp) reconnecting [ 5340.975499] Lustre: *** cfs_fail_loc=159, val=0*** [ 5341.024732] Lustre: Skipped 20 previous similar messages [ 5356.682867] Lustre: lustre-MDT0000: Client 627fc391-d4de-4da8-8bfe-06a854be75fe (at 192.168.201.25@tcp) reconnecting [ 5358.609316] Lustre: 132750:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5365.225601] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 5365.235618] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5365.237640] LustreError: Skipped 1 previous similar message [ 5365.260544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5368.804806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5368.821767] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5368.823639] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5368.851533] LustreError: Skipped 6 previous similar messages [ 5371.257782] Lustre: server umount lustre-OST0000 complete [ 5377.978739] Lustre: server umount lustre-MDT0000 complete [ 5378.022480] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5378.037753] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5378.067947] LustreError: 129732:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5382.487626] LustreError: 132579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773612452 with bad export cookie 3318982899217081470 [ 5382.493568] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5382.501366] LustreError: 132579:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5382.969055] Lustre: server umount lustre-MDT0001 complete [ 5396.383386] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 5400.328194] Key type lgssc unregistered [ 5400.726818] LNet: 133787:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5400.741844] LNetError: 133787:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5400.763815] LNet: Removed LNI 192.168.201.125@tcp [ 5402.101909] Key type .llcrypt unregistered [ 5402.106481] Key type ._llcrypt unregistered [ 5431.194629] Key type ._llcrypt registered [ 5431.204440] Key type .llcrypt registered [ 5431.349939] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 5433.173600] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5433.309708] alg: No test for adler32 (adler32-zlib) [ 5434.599601] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 5434.978258] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 5436.815617] Key type lgssc registered [ 5438.286015] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5447.628816] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5447.650315] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5449.233670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5455.102150] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5459.427283] LustreError: 135186:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5459.441765] LustreError: 135186:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5464.422378] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5464.550149] LustreError: 135187:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5464.859863] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5470.350306] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5477.603458] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5482.867508] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5490.840587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5491.227262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5494.349841] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5498.209697] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5504.687534] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5509.726804] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5515.327683] Lustre: 137436:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5519.220909] Lustre: 137484:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5521.182541] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5522.161437] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5526.504833] Lustre: 137630:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5527.738702] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5528.575642] Lustre: 137728:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5535.214227] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5535.215561] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5535.224128] LustreError: Skipped 1 previous similar message [ 5535.259944] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5535.268915] Lustre: Skipped 1 previous similar message [ 5539.920172] Lustre: server umount lustre-OST0000 complete [ 5545.447948] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5545.474594] Lustre: Skipped 2 previous similar messages [ 5545.480585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5545.506843] Lustre: Skipped 1 previous similar message [ 5550.562968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5550.566988] Lustre: Skipped 1 previous similar message [ 5555.684712] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5555.694406] Lustre: Skipped 1 previous similar message [ 5559.263630] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5559.594281] Lustre: server umount lustre-MDT0000 complete [ 5560.802151] LustreError: 135181:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5560.850880] LustreError: 135181:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5564.199060] LustreError: 135168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773612634 with bad export cookie 17290887449534491301 [ 5564.208127] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5564.208611] LustreError: 135168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5564.566195] Lustre: server umount lustre-MDT0001 complete [ 5576.796608] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 5579.979660] Key type lgssc unregistered [ 5580.393578] LNet: 138775:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5580.406199] LNetError: 138775:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5580.423567] LNet: Removed LNI 192.168.201.125@tcp [ 5581.364181] Key type .llcrypt unregistered [ 5581.370477] Key type ._llcrypt unregistered [ 5608.817321] Key type ._llcrypt registered [ 5608.819760] Key type .llcrypt registered [ 5608.928984] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 5610.032053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5610.237747] alg: No test for adler32 (adler32-zlib) [ 5611.322541] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 5611.606644] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 5613.295175] Key type lgssc registered [ 5614.718303] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5624.804653] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5624.844846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5626.440573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5631.629772] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5636.601323] LustreError: 140176:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5636.633769] LustreError: 140176:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5641.697236] LustreError: 140175:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5642.435856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5642.805742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5648.050510] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5654.903448] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5659.385377] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5668.694509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5669.172648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5671.231524] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5677.421905] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5684.184325] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5687.614673] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5692.951114] Lustre: *** cfs_fail_loc=159, val=0*** [ 5708.419973] Lustre: lustre-MDT0000: Client c88ebfc5-24d7-49b8-9040-5542f6e4571c (at 192.168.201.25@tcp) reconnecting [ 5727.719893] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5727.732424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5727.763216] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5728.747158] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5728.750401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5728.753201] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5728.774813] Lustre: Skipped 1 previous similar message [ 5732.614855] Lustre: server umount lustre-OST0000 complete [ 5738.976731] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5738.981766] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5738.987165] Lustre: Skipped 1 previous similar message [ 5744.100030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5744.110499] Lustre: Skipped 2 previous similar messages [ 5749.216251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5749.230337] Lustre: Skipped 1 previous similar message [ 5751.263295] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5751.537972] Lustre: server umount lustre-MDT0000 complete [ 5754.343493] LustreError: 141245:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5754.375555] LustreError: 141245:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5756.095974] LustreError: 140155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773612826 with bad export cookie 4077942082693431659 [ 5756.097770] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5756.104491] LustreError: 140155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5756.693407] Lustre: server umount lustre-MDT0001 complete [ 5768.400437] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 5771.313398] Key type lgssc unregistered [ 5771.671768] LNet: 143493:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5771.676797] LNetError: 143493:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5772.713950] LNet: Removed LNI 192.168.201.125@tcp [ 5773.829258] Key type .llcrypt unregistered [ 5773.831405] Key type ._llcrypt unregistered [ 5800.979699] Key type ._llcrypt registered [ 5800.982875] Key type .llcrypt registered [ 5801.104610] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 5802.430822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5802.658513] alg: No test for adler32 (adler32-zlib) [ 5803.748574] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 5804.063961] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 5805.759256] Key type lgssc registered [ 5807.618418] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5816.890855] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5816.923334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5818.595373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5823.111851] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5828.086467] LustreError: 144893:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5828.111515] LustreError: 144893:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5832.917325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5833.324977] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5838.444855] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5844.867472] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5849.262345] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5855.811235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5856.216596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5858.287990] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5862.841909] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 5863.414790] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5868.573519] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5888.086256] Lustre: lustre-OST0000: evicting f49c92a5-d990-4b0d-b5bd-bb86b9f46aaf (at 192.168.201.25@tcp) by administrative request [ 5888.100518] Lustre: Skipped 1 previous similar message [ 5919.715491] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5919.724873] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5919.734140] LustreError: Skipped 1 previous similar message [ 5919.754704] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5919.763213] Lustre: Skipped 1 previous similar message [ 5922.365349] Lustre: server umount lustre-OST0000 complete [ 5929.955448] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5929.962680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5929.973739] Lustre: Skipped 2 previous similar messages [ 5929.999019] Lustre: Skipped 1 previous similar message [ 5932.452513] Lustre: server umount lustre-MDT0000 complete [ 5935.083529] LustreError: 144892:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5935.131083] LustreError: 144892:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5936.842322] LustreError: 144872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773613007 with bad export cookie 3690184335626270843 [ 5936.863917] LustreError: 144872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5936.866377] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5937.464803] Lustre: server umount lustre-MDT0001 complete [ 5948.692674] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 5951.787876] Key type lgssc unregistered [ 5952.063877] LNet: 148606:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5952.070267] LNetError: 148606:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5952.099819] LNet: Removed LNI 192.168.201.125@tcp [ 5952.941536] Key type .llcrypt unregistered [ 5952.943730] Key type ._llcrypt unregistered [ 6025.167756] Key type ._llcrypt registered [ 6025.171677] Key type .llcrypt registered [ 6025.314061] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 6026.821656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6026.843071] alg: No test for adler32 (adler32-zlib) [ 6027.914144] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 6028.198360] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 6029.912615] Key type lgssc registered [ 6031.149853] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6039.988994] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6040.001649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6041.696560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6046.028732] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6051.817638] LustreError: 151079:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6051.845285] LustreError: 151079:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 6054.582062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6055.084693] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6059.407259] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6064.567903] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6068.982796] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6077.050512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6077.702932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6080.776369] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 6084.822259] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6086.909070] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 6091.670529] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6102.517432] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6102.519184] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6102.520131] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6102.545835] Lustre: Skipped 1 previous similar message [ 6107.620717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6107.640044] Lustre: Skipped 1 previous similar message [ 6108.200962] Lustre: server umount lustre-OST0000 complete [ 6112.745778] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6112.763783] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6112.769635] Lustre: Skipped 1 previous similar message [ 6112.795885] Lustre: Skipped 1 previous similar message [ 6117.866881] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6117.876582] Lustre: Skipped 1 previous similar message [ 6118.664601] Lustre: server umount lustre-MDT0000 complete [ 6122.134616] LustreError: 151059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773613192 with bad export cookie 838871673271269478 [ 6122.137071] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6122.149015] LustreError: 151059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6122.463871] Lustre: server umount lustre-MDT0001 complete [ 6134.559622] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 6137.722213] Key type lgssc unregistered [ 6138.059101] LNet: 154270:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6138.081293] LNetError: 154270:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6138.101633] LNet: Removed LNI 192.168.201.125@tcp [ 6139.309405] Key type .llcrypt unregistered [ 6139.312487] Key type ._llcrypt unregistered [ 6167.218947] Key type ._llcrypt registered [ 6167.225037] Key type .llcrypt registered [ 6167.424123] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 6169.070411] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6169.102087] alg: No test for adler32 (adler32-zlib) [ 6170.386839] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 6170.753715] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 6172.472446] Key type lgssc registered [ 6173.986445] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6190.841408] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:20:58 (1773613258) [ 6206.537324] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [ 6215.619222] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 6252.053802] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6252.112980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6253.432846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6253.475945] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6253.634356] Lustre: lustre-MDT0000: new disk, initializing [ 6253.737312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6253.756562] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6258.131789] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6268.751474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6268.912946] Lustre: 158502:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6268.970435] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6268.978460] Lustre: Skipped 1 previous similar message [ 6269.046944] Lustre: lustre-MDT0001: new disk, initializing [ 6269.141527] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6269.194488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6269.208880] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6273.514802] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6281.051934] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6284.632051] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6291.735870] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6292.216965] Lustre: lustre-OST0000: new disk, initializing [ 6292.251125] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6292.458292] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6299.926915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6299.955341] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6300.719309] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6303.760378] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6321.404533] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6342.115353] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6342.132039] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6342.144116] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6342.155538] Lustre: Skipped 1 previous similar message [ 6342.179605] Lustre: Skipped 1 previous similar message [ 6344.685874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6344.702140] Lustre: Skipped 1 previous similar message [ 6348.823660] Lustre: server umount lustre-OST0000 complete [ 6357.474267] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6357.495074] LustreError: Skipped 1 previous similar message [ 6357.509902] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6357.549319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6360.033156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6360.033774] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6364.358705] Lustre: server umount lustre-MDT0000 complete [ 6371.595518] LustreError: 158493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773613441 with bad export cookie 11926725256902981025 [ 6371.603155] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6371.607015] LustreError: 158493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6372.696103] Lustre: server umount lustre-MDT0001 complete [ 6389.916554] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 6394.439868] Key type lgssc unregistered [ 6395.026186] LNet: 162023:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6395.039584] LNetError: 162023:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6395.088812] LNet: Removed LNI 192.168.201.125@tcp [ 6396.546969] Key type .llcrypt unregistered [ 6396.549872] Key type ._llcrypt unregistered [ 6434.271844] Key type ._llcrypt registered [ 6434.275666] Key type .llcrypt registered [ 6434.438534] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 6436.319192] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6436.362181] alg: No test for adler32 (adler32-zlib) [ 6438.030891] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 6438.383326] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 6440.119185] Key type lgssc registered [ 6442.607103] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6454.575110] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6454.626255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6456.424353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6463.670941] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6466.533214] LustreError: 163423:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6466.586482] LustreError: 163423:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 6471.650481] LustreError: 163424:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6476.461357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6476.769831] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 6476.950671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6484.222304] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6491.496044] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6497.299716] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6509.304338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6509.823157] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6520.010411] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 6526.222418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6528.660056] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6537.043205] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6537.067846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6537.099846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6537.110843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.136654] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6538.542395] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6538.569783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6538.593670] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6538.610504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.632487] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6540.263871] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6540.298566] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6540.324825] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6540.339256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.356714] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6541.862904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.192059] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6543.217878] Lustre: Skipped 1 previous similar message [ 6543.259433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6543.281542] Lustre: Skipped 1 previous similar message [ 6543.289311] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6543.302915] Lustre: Skipped 1 previous similar message [ 6543.310783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.327713] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6543.343766] Lustre: Skipped 1 previous similar message [ 6544.559883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.087658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.502814] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6547.507825] Lustre: Skipped 2 previous similar messages [ 6547.532944] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6547.553545] Lustre: Skipped 2 previous similar messages [ 6547.566933] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6547.581703] Lustre: Skipped 2 previous similar messages [ 6547.592646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.615652] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6547.644420] Lustre: Skipped 2 previous similar messages [ 6548.865672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.286992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.615836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.044189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.344445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.861570] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6555.882363] Lustre: Skipped 5 previous similar messages [ 6555.906771] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6555.947855] Lustre: Skipped 5 previous similar messages [ 6555.956303] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6555.966937] Lustre: Skipped 5 previous similar messages [ 6555.975601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.990460] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6556.001708] Lustre: Skipped 5 previous similar messages [ 6557.435455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.860185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.330348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.718961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.969394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.525554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.710491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.055561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.510445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.877542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.460983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.814848] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6572.829687] Lustre: Skipped 11 previous similar messages [ 6572.849049] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6572.867429] Lustre: Skipped 11 previous similar messages [ 6572.883181] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6572.895383] Lustre: Skipped 11 previous similar messages [ 6572.904464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.932968] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6572.958529] Lustre: Skipped 11 previous similar messages [ 6574.408966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.714330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.926896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.151804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.438186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.639445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.918665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.341908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.658431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.870225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.157236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.294273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.946297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.337915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.010178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.597643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.831185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.269234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.264850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.995253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.559826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.983582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.445379] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6605.473649] Lustre: Skipped 22 previous similar messages [ 6605.508543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6605.545518] Lustre: Skipped 22 previous similar messages [ 6605.557801] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6605.575609] Lustre: Skipped 22 previous similar messages [ 6605.588588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.615101] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6605.629401] Lustre: Skipped 22 previous similar messages [ 6607.442263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.183470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.920950] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6610.929047] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6610.948781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.710812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.269812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.991916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.386919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.655830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.824951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.907509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.330832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.647764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.578841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.923945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.277454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.886239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.025725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.224860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.370296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.889221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.246539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.562131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.847123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.380479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.501561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.823543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.143343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.457549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.684393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.033896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.323152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.019412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.164750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.690432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.221290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.603197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.024305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.461121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.823771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.150765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.500167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.815192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.887720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.103917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.311555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.700982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.953228] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6669.960995] Lustre: Skipped 46 previous similar messages [ 6670.007624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6670.029804] Lustre: Skipped 46 previous similar messages [ 6670.039881] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6670.056237] Lustre: Skipped 46 previous similar messages [ 6670.073752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.108446] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6670.132105] Lustre: Skipped 46 previous similar messages [ 6671.827463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.130219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.501462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.861534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.224215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.808673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.376809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.712751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.956706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.350526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.584667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.472866] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6687.506136] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6687.511401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.154405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.647972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.267221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.563947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.982469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.618694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.115670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.564516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.903443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.426375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.143235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.596972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.152777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.240022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.439382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.757131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.967555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.364367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.855531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.497735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.833587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.161457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.578580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.265603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.670021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.279814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.663936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.863674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.371265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.768736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.141860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.528526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.899627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.295996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.775253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.292589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.804568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.270911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.712706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.447575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.961670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.209555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.837705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.463856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.994747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.604513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.256243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.734088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.512351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.035858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.460221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.940050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.389776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.698244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.934034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.421230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.876129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.401243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.822593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.334700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.777708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.318126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.152776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.795160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.111873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.600052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.783389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.193764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.749633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.039563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.421894] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6791.432986] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6791.453190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.087418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.503508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.886369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.369806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.013510] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6799.020902] Lustre: Skipped 87 previous similar messages [ 6799.052985] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6799.070902] Lustre: Skipped 87 previous similar messages [ 6799.079579] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6799.085344] Lustre: Skipped 87 previous similar messages [ 6799.095518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.114902] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6799.124438] Lustre: Skipped 87 previous similar messages [ 6800.262059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.503413] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6801.526290] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6801.532669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.912259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.456571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.838274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.339194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.946476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.424989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.734681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.113517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.525959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.873729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.225887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.358665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.709982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.048237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.708770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.167204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.687570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.144487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.500293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.982877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.393832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.769251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.217553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.008645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.208572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.768802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.528582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.130339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.237463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.515473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.107519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.590436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.118287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.122468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.117883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.755973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.623252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.456612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.317848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.273503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.071134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.419813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.998864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.168677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.132762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.697901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.960282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.854510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.941913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.382540] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6873.390630] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6873.407951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.174824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.686550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.646530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.890548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.228661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.538287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.282800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.936228] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6884.958690] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6884.986916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.787532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.528380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.094399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.903156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.266570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.545568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.168858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.836689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.231665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.598890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.017663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.377642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.939810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.497985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6907.849354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6909.031608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.578583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6912.169950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6913.911955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.737751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.111051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.676550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.229284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.695688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6923.189205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.923540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.414641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.748117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.081960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.596224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.160706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.522647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.143108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.622460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.895762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.357616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.874754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6942.330361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6943.804741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6945.435851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6947.172831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6948.503325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6949.978369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6951.474681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.971334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6954.290846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6955.787783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.156327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.727546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6960.493411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6961.849203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6963.162686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.975588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6966.824855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6968.276667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6969.762929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6971.254905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.862423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.281709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.914991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6977.545941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6978.862116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.525627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6981.920211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.545795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.031991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6986.520230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6988.091726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6989.347386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.887935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.483377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.827404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6995.498905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6996.978955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.817132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7000.447178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.159955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7003.813508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7005.461368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7006.865954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7008.578871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7010.510562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.951232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7013.487645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7014.940541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.494040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7018.212653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7019.494378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.161703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7022.732898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7024.228200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7025.854809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7027.530801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.048032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7030.630426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7032.012968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7033.440644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7034.945964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7036.453609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7037.645891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.338803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7040.642966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7042.317222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7043.810103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7045.212614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7046.976158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7048.659395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7050.611186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7051.990155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7053.951796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7056.205658] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7056.232868] Lustre: Skipped 170 previous similar messages [ 7056.285500] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7056.315360] Lustre: Skipped 170 previous similar messages [ 7056.329579] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7056.346034] Lustre: Skipped 170 previous similar messages [ 7056.354684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7056.375773] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7056.398507] Lustre: Skipped 170 previous similar messages [ 7058.353253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.126432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7061.840099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7063.562193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7064.898554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7066.605253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7068.146514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7069.674222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7070.996544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.631570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7074.127549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.690400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7077.252174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.701728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.267148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7082.080791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7084.046992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7085.710085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.259900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7088.759073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7090.454666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7092.057924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7093.951390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7095.906121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7097.546393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7098.963654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7100.274331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7101.927712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7102.990534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7104.793357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.684832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7108.276358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7109.723158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7111.341822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7112.830647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7114.241657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7115.590698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7117.106524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.646432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.263483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7121.997704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.369182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.766082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7126.254332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.901558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.881671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7131.364150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7132.941328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7134.482468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.801310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.181103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7138.581599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.890851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7141.236406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7142.860125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.267609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7145.616537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7147.097083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7148.474490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7150.103606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7151.629855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.311400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7154.692059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7155.980862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.350307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7158.844293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7160.359041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7161.812583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7163.216466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.967887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7166.435267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.782094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7169.139656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7170.706625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7172.336970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7173.802969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7175.301095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7176.963360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7178.450682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7180.129606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7181.722642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.955789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7184.359646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7185.683272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7187.524618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7189.233482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7190.770279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7192.078311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.481422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7195.188415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7196.566182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.154527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7199.768823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7201.297746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7202.494701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7203.983700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7205.313152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.773778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7208.133212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7209.811684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7211.664498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7213.176432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7214.603623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7216.247030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7217.825687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7219.386789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7220.921312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7222.481828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7223.992876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7225.814265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.275351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.791081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7230.176253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.460834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7232.583333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.146987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7235.599320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7236.988360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.370264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7239.791465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7241.269312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7243.103188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.805142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.287483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.821837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7249.475075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7250.728079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7252.229828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7253.634446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7255.098048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7256.435847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7257.661920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7259.263765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7260.845109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.306960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7263.627434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7265.497792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7266.805602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7268.486263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7270.291628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7272.070257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7273.218900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7274.743865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7276.256249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.983839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7279.438739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7281.046165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7282.480946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7283.960102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7285.834141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7287.475325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.925290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7290.486691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7291.959139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7293.393987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7294.669113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7296.472537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.919319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7299.789611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7301.308553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7303.141277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7304.757415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.485912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7308.201142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7309.896938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7311.234598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7312.684731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.937339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7315.809259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7317.355106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7318.997863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7320.615267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.052425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.890365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7325.499295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.086852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7328.482613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.830595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7331.113701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7332.404862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.059833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7335.575722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.055398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7338.563355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7340.324074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.108655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7343.830914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7345.236416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7346.926382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7348.416674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7349.780510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7351.250610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.510802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.106346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7355.426359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7357.152537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7358.668611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7360.069942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7361.480147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7363.022765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.544073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.177766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7367.492182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7369.145325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.578865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7372.046309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7373.742877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7375.683071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7377.116494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.745777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7380.513443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7382.268965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7383.663580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.410091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7386.910654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7388.543550] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7388.568377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7388.569195] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7390.188374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7391.499550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.717963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7394.229930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7396.058807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7397.762597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.024487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7400.378807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7401.676158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7402.969662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7404.147724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7405.534547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7406.783631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.338249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7409.930384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7411.194830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7412.604250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7414.055263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7415.697292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.167971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7418.421127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7419.619273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.027049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7422.425432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7423.793986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7425.380288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.797635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.145180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.565646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7431.267576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7432.856558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7434.345325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7435.881981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7437.481428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7438.934376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7440.377053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7442.036899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7443.777863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.197995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7446.451289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.666730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7449.031193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7450.552731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7451.688995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7453.087354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7454.349845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7455.624330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7457.009891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7458.888574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.283268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7461.669319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7462.608450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7464.662474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7466.490351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7468.002488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7469.168565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7470.498142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7472.032624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7473.541499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7475.441240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7477.503169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7479.604633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.507499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7483.256423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7485.063295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.734982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7488.975316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7490.605122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.130980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.907848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.326325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7496.737867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7497.837443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7499.220078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7500.509758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7502.068329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7503.403336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7504.803782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.333289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7507.772588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.807721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7510.115925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7511.473560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7512.832506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7514.337406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7515.647648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7517.294913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7518.500572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7519.798955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7521.041505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7522.499494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7523.893470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7525.217083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7526.686294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7527.996508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7529.335525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7530.569686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7531.850768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7533.421450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7534.899633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7536.342727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.692743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7538.890782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.310521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7541.445268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7543.024317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7544.980160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7546.368480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7547.972158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7549.405623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7550.991630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7552.480707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7554.098075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7555.663200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.755055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7558.174716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7559.528700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7560.819106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7562.003570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7563.435402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7564.856728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7566.289107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7567.690486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7569.113670] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7569.119491] Lustre: Skipped 339 previous similar messages [ 7569.152705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7569.163928] Lustre: Skipped 339 previous similar messages [ 7569.168961] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7569.175181] Lustre: Skipped 339 previous similar messages [ 7569.179629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7569.191523] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7569.206619] Lustre: Skipped 339 previous similar messages [ 7570.640766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7572.112378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7573.630779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7574.967143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7576.511437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.625412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7578.822157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7580.066401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7581.356834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.570132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7583.778419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7585.148595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.196356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7587.160289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7588.285901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7589.766806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7590.650380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7591.948145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7593.388916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7594.640603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7596.379292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7598.148697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7599.556857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7600.983754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7602.349830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.103595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7605.656473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7607.090450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7608.505062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7609.984904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7611.506529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7612.964215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7614.390520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7615.935074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7617.469032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7619.092192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7620.833032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.344654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7623.994216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7625.554579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7626.980868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7628.482220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7629.764960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7631.038739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7632.492214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7633.984051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7635.445815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7636.724171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7637.978396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7639.117563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7640.676238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7642.147275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7643.540304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7645.005609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.729677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7648.524605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7650.079979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7651.480308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7652.955624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7654.563436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7655.970471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7657.431181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7658.641280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7660.172608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7661.755675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7663.234200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7665.045058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7666.625233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7667.841245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7669.251554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7670.720063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7672.214460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7673.655394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7675.133647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7676.885930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.359681] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7678.370055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.370865] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7679.392053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7680.643737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7682.256597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7683.810293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7685.293636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7686.581291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7688.012645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7689.490239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7690.815695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7692.544872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7694.337301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7696.184659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7697.746091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7699.372364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7701.158886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7702.499708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7703.904432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7705.277615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7706.759656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7708.092307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7709.474169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7710.974904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7712.426913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7713.688676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7714.664713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7715.746787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.962250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7718.775206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7720.682885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7723.042358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7725.438658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7727.242214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7728.858896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7730.170482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7731.525068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7732.982061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7734.414396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7736.024264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7737.426899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7738.897640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7740.502821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7742.025665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7743.378941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7744.879754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7746.500327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7747.850780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7749.180907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7750.756534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7752.373841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7753.606048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7754.997785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7756.353371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7757.895936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7759.173353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7760.534551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7761.901179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7763.429508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7764.568583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7765.990515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7767.257756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7768.974877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7770.406289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7771.850899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7773.247580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7774.525812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7775.809109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7777.189031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7778.781468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7780.210951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7781.523265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7783.001936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7784.862736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7786.715453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7788.371596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7789.789952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7791.180862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7792.430629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7793.773437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7795.410942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7796.685823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7797.968963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7799.199966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7800.138989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7801.237473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7802.720318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7804.103861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7805.288725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7806.596478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7807.826500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7808.983141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7810.245172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7811.617549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7813.009713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7814.499481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7815.798747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7817.258356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.709078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7820.045562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7821.552231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7823.075410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7824.533480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7826.112552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7827.382789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7828.663140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7829.867395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7831.277946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7832.915813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7834.145254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7835.540515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7837.124074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7838.521807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7839.793228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7841.118364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7842.629290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7843.968659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7845.289782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7846.927432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7848.365566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7849.886954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7851.018383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7852.610505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7854.094153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7855.565092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7857.221453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7858.335619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7859.949225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7861.455826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7863.054883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7864.446408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7865.793847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7867.164088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7868.868348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7870.470568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7871.813541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7873.291318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7874.682910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7876.240960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.649067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7879.238957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7880.329437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.534823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7882.758380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7883.780645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7885.452946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7886.755733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7888.571226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7890.060548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.691058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7893.649881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7895.139905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7896.976371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7899.577325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7901.269773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7902.921369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7904.331216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7905.784237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7907.113099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7908.623595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7909.766378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7911.443687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7912.918293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7914.224332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7915.500760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7916.835400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7918.046581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7919.385230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7920.654191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7922.009587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7923.252091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7924.251797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7925.661851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7927.063549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7928.735143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7930.050478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7931.331171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7932.617567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7933.702600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7935.090212] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7935.119176] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 7935.139771] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7935.143705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7935.150053] LustreError: 163443:0:(osp_precreate.c:1355:osp_precreate_thread()) Skipped 1 previous similar message [ 7936.620589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7938.045029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7939.644599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7941.320275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.725260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7944.314660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7945.801201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7947.173943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7948.699058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7949.653481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7950.938360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7952.376186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7953.765526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7955.260869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7956.545922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7958.212772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7959.774645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7961.166481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7962.371236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7963.688731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7965.066986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7966.559124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7968.484918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7970.038674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7971.535851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7972.718364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7974.371524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7975.803222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7977.295459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7979.170599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7980.758213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7982.258206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7983.746146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7985.160838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7986.859478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7988.254973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7989.841590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7991.353909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7992.747663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7993.979524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7995.363035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7996.657939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7998.061548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7999.558383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8001.030902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8002.531654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8003.802154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8005.009791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8006.217160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8007.465832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8008.953912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8660.349737] LustreError: 165023:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 8660.363479] LustreError: 163443:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 9575.908842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9575.937476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9575.941957] Lustre: Skipped 306 previous similar messages [ 9575.966249] Lustre: Skipped 1 previous similar message [ 9580.910751] Lustre: server umount lustre-OST0000 complete [ 9589.729962] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9589.740103] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9591.776648] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9593.181445] Lustre: server umount lustre-MDT0000 complete [ 9595.874099] LustreError: 164263:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9595.901117] LustreError: 164263:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 9598.805264] LustreError: 163403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773616668 with bad export cookie 1976608460278387636 [ 9598.809034] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9598.834057] LustreError: 163403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9599.566177] Lustre: server umount lustre-MDT0001 complete [ 9615.350807] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 9620.103974] Key type lgssc unregistered [ 9620.544880] LNet: 218494:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9620.559456] LNetError: 218494:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9620.617365] LNet: Removed LNI 192.168.201.125@tcp [ 9621.763917] Key type .llcrypt unregistered [ 9621.769708] Key type ._llcrypt unregistered [ 9651.938392] Key type ._llcrypt registered [ 9651.939777] Key type .llcrypt registered [ 9652.109432] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 9653.909226] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9653.982360] alg: No test for adler32 (adler32-zlib) [ 9655.161744] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 9655.427952] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 9657.128463] Key type lgssc registered [ 9659.132698] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9669.496399] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9669.542390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9671.484362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9676.879780] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9680.879096] LustreError: 219897:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9680.912670] LustreError: 219897:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 9686.010681] LustreError: 219896:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9687.484449] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9687.878104] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9693.482719] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9700.419724] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9705.575483] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9714.741306] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9715.203269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9716.239033] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 9723.053343] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9729.293275] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9737.719656] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 9739.912425] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9739.925878] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 9739.935162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9739.959353] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9739.960820] LustreError: 219916:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 9742.820844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9742.833603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9742.840564] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9746.523763] Lustre: lustre-OST0000: Not available for connect from 192.168.201.25@tcp (stopping) [ 9746.532134] Lustre: Skipped 1 previous similar message [ 9747.948345] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9747.966532] Lustre: Skipped 1 previous similar message [ 9748.593213] Lustre: server umount lustre-OST0000 complete [ 9760.031486] LustreError: 219916:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 9785.624038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9786.115500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9794.461877] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9798.394605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9798.408887] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9798.410163] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9798.436426] Lustre: Skipped 1 previous similar message [ 9801.440577] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9803.232762] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 9808.291545] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b8fd19bf000.ost_server_uuid 50 [ 9809.742530] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b8fd19bf000.ost_server_uuid in IDLE state after 0 sec [ 9815.523788] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9815.535334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9815.555594] Lustre: Skipped 1 previous similar message [ 9815.564747] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9819.110573] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9819.124110] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9821.464389] Lustre: server umount lustre-OST0000 complete [ 9826.273721] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9826.287228] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9826.311878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9826.323393] Lustre: Skipped 2 previous similar messages [ 9834.474762] LustreError: 219893:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9834.500617] LustreError: 219893:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 9834.572541] Lustre: server umount lustre-MDT0000 complete [ 9839.076043] LustreError: 219876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773616909 with bad export cookie 12823896540893588410 [ 9839.080537] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9839.097043] LustreError: 219876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9839.485408] Lustre: server umount lustre-MDT0001 complete [ 9854.412903] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [ 9858.571483] Key type lgssc unregistered [ 9858.845485] LNet: 224548:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9858.852473] LNetError: 224548:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9858.868637] LNet: Removed LNI 192.168.201.125@tcp [ 9860.067693] Key type .llcrypt unregistered [ 9860.073589] Key type ._llcrypt unregistered [ 9902.310685] Key type ._llcrypt registered [ 9902.319218] Key type .llcrypt registered [ 9902.597302] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [ 9904.531508] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9904.613786] alg: No test for adler32 (adler32-zlib) [ 9906.152672] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [ 9906.488508] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [ 9908.255213] Key type lgssc registered [ 9909.975519] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9922.173372] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9922.209496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9924.052669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9929.702457] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9934.305066] LustreError: 226265:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9934.337184] LustreError: 226265:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 9939.430729] LustreError: 226264:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9940.850114] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9941.363468] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9946.894489] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9950.606777] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9959.216061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9959.788381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9967.659594] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9969.038425] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 9977.971700] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9978.272512] Lustre: lustre-OST0001: new disk, initializing [ 9978.279383] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9978.432076] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9985.601239] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [ 9986.142687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9986.158267] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9986.275413] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9998.381566] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10006.304936] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10027.508508] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10027.523800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10027.536849] Lustre: Skipped 2 previous similar messages [10027.550013] Lustre: Skipped 3 previous similar messages [10032.618427] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10032.650234] Lustre: Skipped 3 previous similar messages [10034.027761] Lustre: server umount lustre-MDT0000 complete [10037.761273] LustreError: 226261:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10037.783780] LustreError: 226261:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [10038.558135] LustreError: 226243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773617108 with bad export cookie 5719010526662769959 [10038.559421] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10038.573774] LustreError: 226243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10038.925615] Lustre: server umount lustre-MDT0001 complete [10053.918194] Lustre: server umount lustre-OST0000 complete [10068.703846] Lustre: server umount lustre-OST0001 complete [10095.490189] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10107.977310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10108.602340] LustreError: 231602:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10108.763912] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10113.724556] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10114.025591] LustreError: 231603:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10123.247296] LustreError: 231602:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10123.283810] LustreError: 231602:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [10124.189693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10124.632957] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10129.531661] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10132.655639] Lustre: 232711:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10140.785559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10141.378400] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10148.847986] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10149.552458] LustreError: 233065:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10149.582290] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [10153.661474] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [10159.817256] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10167.279859] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10176.307220] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10180.409699] Lustre: 234555:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10201.057374] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10201.071493] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10201.078057] Lustre: Skipped 3 previous similar messages [10201.108654] Lustre: Skipped 3 previous similar messages [10206.177795] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10211.301565] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10211.307231] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [10211.317455] Lustre: Skipped 6 previous similar messages [10211.610838] Lustre: server umount mylustre-MDT0000 complete [10216.151798] LustreError: 231578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773617286 with bad export cookie 5719010526662772822 [10216.167599] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10216.174618] LustreError: 231578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10216.422401] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10216.433712] LustreError: 232773:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10216.442417] Lustre: Skipped 1 previous similar message [10216.490467] LustreError: 232773:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [10216.931923] Lustre: server umount mylustre-MDT0001 complete [10222.228258] Lustre: server umount mylustre-OST0000 complete [10227.377755] Lustre: server umount mylustre-OST0001 complete [10255.323595] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10268.895668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10269.436062] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10269.443175] Lustre: Skipped 1 previous similar message [10275.619511] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10284.004357] LustreError: 236854:0:(ldlm_lib.c:1178:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10284.032256] LustreError: 236854:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [10287.582174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10294.375947] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10298.367225] Lustre: 237964:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10306.389630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10313.026596] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [10314.054217] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [10315.596558] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10326.519558] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10335.846581] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10344.895419] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10349.385769] Lustre: 239810:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10362.850515] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10362.874315] Lustre: Skipped 2 previous similar messages [10362.890223] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [10362.900672] Lustre: Skipped 2 previous similar messages [10368.797710] Lustre: server umount tfs-MDT0000 complete [10373.015774] LustreError: 238344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773617443 with bad export cookie 5719010526662774782 [10373.016184] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10373.027298] LustreError: 238344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10373.092191] Lustre: tfs-MDT0001-lwp-OST0000: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10373.105699] Lustre: Skipped 1 previous similar message [10373.406909] Lustre: server umount tfs-MDT0001 complete [10379.936928] Lustre: server umount tfs-OST0000 complete [10384.635927] Lustre: server umount tfs-OST0001 complete [10409.900466] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10422.358518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10423.014950] LustreError: 242044:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10423.041957] LustreError: 242044:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 10 previous similar messages [10423.135955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10423.146354] Lustre: Skipped 3 previous similar messages [10428.747868] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10440.127321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10446.659723] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10450.709714] Lustre: 243152:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10458.857432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10465.611545] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [10468.146968] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10468.703366] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [10480.127144] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10489.449584] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10498.597596] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10502.667941] Lustre: 244998:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10515.200145] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:33:02 (1773617582) [10531.810343] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10531.817189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10531.825682] Lustre: Skipped 3 previous similar messages [10531.842483] Lustre: Skipped 11 previous similar messages [10537.799889] Lustre: server umount lustre-MDT0000 complete [10541.440811] LustreError: 242021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773617611 with bad export cookie 5719010526662776476 [10541.447721] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10541.453834] LustreError: 242021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10547.909628] Lustre: server umount lustre-MDT0001 complete [10551.466812] Lustre: server umount lustre-OST0000 complete [10554.802902] Lustre: server umount lustre-OST0001 complete [10561.015713] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [10571.180337] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10604.705864] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [10604.705864] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10604.705864] [10604.729801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10605.015469] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10605.052654] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10605.125382] Lustre: lustre-MDT0000: new disk, initializing [10605.204144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10605.215422] Lustre: Skipped 3 previous similar messages [10605.231418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10609.877961] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10620.118541] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [10620.118541] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10620.118541] [10620.187295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10620.347510] Lustre: 248149:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10620.403973] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10620.408756] Lustre: Skipped 1 previous similar message [10620.558929] Lustre: lustre-MDT0001: new disk, initializing [10620.781412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10620.785991] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10625.358585] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10634.521717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10634.797187] Lustre: lustre-OST0000: new disk, initializing [10634.801962] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10636.178746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10636.192978] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10636.379620] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10640.915469] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10650.856411] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10651.026341] Lustre: lustre-OST0001: new disk, initializing [10651.031289] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10652.131691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10652.145362] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10652.273571] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10657.203938] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10667.238968] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:35:34 (1773617734) [10679.637729] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:35:47 (1773617747) [10697.697669] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10697.707489] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10697.720354] Lustre: Skipped 3 previous similar messages [10697.731465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10697.736735] Lustre: Skipped 9 previous similar messages [10699.395460] Lustre: server umount lustre-OST0000 complete [10702.817571] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10702.826140] LustreError: Skipped 1 previous similar message [10703.352563] LustreError: 249679:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10703.362654] LustreError: 249679:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 14 previous similar messages [10709.006578] Lustre: server umount lustre-MDT0000 complete [10711.639885] LustreError: 248141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773617781 with bad export cookie 5719010526662778023 [10711.642172] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10711.646047] LustreError: 248141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10717.965823] Lustre: server umount lustre-MDT0001 complete [10725.566440] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [10748.383213] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10748.504604] Lustre: server umount lustre-OST0001 complete [10752.718496] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [10763.936470] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10791.675699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10791.882908] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10791.920589] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10791.983330] Lustre: lustre-MDT0000: new disk, initializing [10792.061284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10794.682646] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10802.820323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10802.926286] Lustre: 255020:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10802.980769] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10802.984901] Lustre: Skipped 1 previous similar message [10803.055316] Lustre: lustre-MDT0001: new disk, initializing [10803.155232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10803.165034] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10806.086025] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10812.326891] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10815.078201] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10821.305331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10821.583430] Lustre: lustre-OST0000: new disk, initializing [10821.590245] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10823.602903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10823.612754] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10823.734482] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10827.321319] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10833.289778] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10839.010706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10839.026520] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10839.032636] Lustre: Skipped 5 previous similar messages [10839.041508] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10839.041515] Lustre: Skipped 11 previous similar messages [10843.671886] Lustre: server umount lustre-OST0000 complete [10849.250532] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10852.558403] Lustre: server umount lustre-MDT0000 complete [10855.332156] LustreError: 255012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773617925 with bad export cookie 5719010526662779500 [10855.337328] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10855.343276] LustreError: 255012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10855.560870] Lustre: server umount lustre-MDT0001 complete [10863.294226] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [10865.355631] Key type lgssc unregistered [10865.553882] LNet: 258388:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10865.561736] LNetError: 258388:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10865.571436] LNet: Removed LNI 192.168.201.125@tcp [10866.230276] Key type .llcrypt unregistered [10866.233588] Key type ._llcrypt unregistered [10885.680569] Key type ._llcrypt registered [10885.683409] Key type .llcrypt registered [10885.776963] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10886.635656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10886.658764] alg: No test for adler32 (adler32-zlib) [10887.706946] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [10887.953901] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [10889.617288] Key type lgssc registered [10890.586772] Lustre: Echo OBD driver; http://www.lustre.org/ [10896.495629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10896.525741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10897.886760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10901.036871] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10907.554183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10907.817787] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10910.298835] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10913.940639] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10915.906108] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10920.040501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10920.348201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10924.481397] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [10924.903749] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [10928.736257] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10929.418541] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [10932.708887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10932.708937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10936.891378] Lustre: server umount lustre-OST0000 complete [10942.949755] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10942.951203] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10942.962166] Lustre: Skipped 1 previous similar message [10942.962743] Lustre: Skipped 1 previous similar message [10945.276567] Lustre: server umount lustre-MDT0000 complete [10947.606977] LustreError: 259792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773618017 with bad export cookie 11742208249572274171 [10947.612312] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10947.614241] LustreError: 259792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10947.828181] Lustre: server umount lustre-MDT0001 complete [10953.985240] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [10955.756180] Key type lgssc unregistered [10955.973988] LNet: 263046:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10955.984686] LNetError: 263046:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10956.002500] LNet: Removed LNI 192.168.201.125@tcp [10956.532442] Key type .llcrypt unregistered [10956.533279] Key type ._llcrypt unregistered [10966.568655] Key type ._llcrypt registered [10966.569777] Key type .llcrypt registered [10966.620475] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [10967.258653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10967.271750] alg: No test for adler32 (adler32-zlib) [10968.221405] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [10968.389976] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [10970.063419] Key type lgssc registered [10970.780030] Lustre: Echo OBD driver; http://www.lustre.org/ [10975.960133] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10975.970222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10991.455340] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [10996.575417] LustreError: 264199:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.125@tcp: failed processing log, type 4: rc = -110 [11023.263379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11026.534016] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11029.781098] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11030.989927] Lustre: server umount lustre-OST0000 complete [11040.346640] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [11042.132147] Key type lgssc unregistered [11042.347881] LNet: 265701:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11042.353608] LNetError: 265701:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11042.365297] LNet: Removed LNI 192.168.201.125@tcp [11042.828060] Key type .llcrypt unregistered [11042.831695] Key type ._llcrypt unregistered [11065.203315] Key type ._llcrypt registered [11065.205555] Key type .llcrypt registered [11065.296527] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [11066.097063] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11066.141115] alg: No test for adler32 (adler32-zlib) [11067.089897] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [11067.244313] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [11068.863202] Key type lgssc registered [11069.487814] Lustre: Echo OBD driver; http://www.lustre.org/ [11077.300708] loop0: detected capacity change from 0 to 204800000 [11077.396544] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11077.410388] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11077.599309] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11078.150123] loop0: detected capacity change from 0 to 204800000 [11078.244512] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11078.256108] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11078.454954] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11078.990130] loop0: detected capacity change from 0 to 204800000 [11079.092934] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11079.100886] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11079.175890] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11079.782617] loop0: detected capacity change from 0 to 204800000 [11079.894736] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11080.632675] loop0: detected capacity change from 0 to 204800000 [11080.658182] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11083.206971] loop0: detected capacity change from 0 to 204800000 [11083.228067] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11085.493614] loop0: detected capacity change from 0 to 204800000 [11085.509411] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11087.868908] loop0: detected capacity change from 0 to 204800000 [11087.885572] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11089.923158] loop0: detected capacity change from 0 to 204800000 [11089.956678] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11089.972845] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11090.015794] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11090.022140] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [11090.029073] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [11090.039413] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [11090.046907] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [11090.054921] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [11090.060826] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [11090.067261] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [11090.074512] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [11090.082033] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [11090.087427] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [11090.094799] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [11090.109697] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [11090.123747] Lustre: 269073:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [11093.649804] Lustre: server umount MGS complete [11094.134822] loop0: detected capacity change from 0 to 204800000 [11094.173037] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11094.448571] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11094.478462] Lustre: lustre-MDT0000: Aborting client recovery [11094.946809] loop1: detected capacity change from 0 to 204800000 [11094.984764] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11095.031237] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11095.039761] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [11095.046851] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [11095.053064] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [11095.061925] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [11095.067975] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [11095.075205] Lustre: 269505:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [11095.089150] Lustre: Found index 1 for lustre-MDT0001, updating log [11095.229686] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11095.266048] Lustre: lustre-MDT0001: Aborting client recovery [11095.857195] loop2: detected capacity change from 0 to 204800000 [11095.934487] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11095.988869] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11095.995690] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11096.001191] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [11096.014768] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [11096.023981] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [11096.037371] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11096.048231] Lustre: 269586:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11096.145820] Lustre: Found index 0 for lustre-OST0000, updating log [11096.225042] Lustre: lustre-OST0000: Aborting recovery [11096.720980] loop3: detected capacity change from 0 to 204800000 [11096.759481] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11096.810547] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11096.819780] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11096.825631] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [11096.832229] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [11096.837991] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [11096.844595] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11096.850799] Lustre: 269670:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11096.914053] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11096.919927] Lustre: Skipped 1 previous similar message [11096.931152] Lustre: lustre-OST0001: Aborting recovery [11102.192002] LustreError: 269449:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [11104.227186] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11104.227686] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11104.241988] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [11104.245321] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [11105.255714] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [11105.256037] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [11109.431381] LustreError: 269416:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 636us: rc = -2 [11121.119838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11121.124230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11121.130825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11125.729997] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11125.730808] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11125.733382] Lustre: Skipped 2 previous similar messages [11125.739118] Lustre: Skipped 2 previous similar messages [11127.432688] Lustre: server umount lustre-MDT0000 complete [11127.937022] LustreError: 269399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773618198 with bad export cookie 2071434418370746323 [11127.941270] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11127.948703] LustreError: 269399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11130.847521] LustreError: 269414:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11130.847867] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11130.849760] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11130.849767] Lustre: Skipped 1 previous similar message [11130.854956] LustreError: 269414:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [11130.862213] Lustre: Skipped 1 previous similar message [11132.897222] LustreError: 269413:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11132.897535] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11132.911284] LustreError: 269413:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11132.919197] Lustre: Skipped 2 previous similar messages [11134.188635] Lustre: server umount lustre-MDT0001 complete [11149.279099] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11149.339508] Lustre: server umount lustre-OST0000 complete [11164.128325] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11164.232798] Lustre: server umount lustre-OST0001 complete [11169.051136] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 19:43:57 (1773618237) [11184.332878] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [11190.113515] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [11212.629021] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11212.814091] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11212.844118] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11212.899880] Lustre: lustre-MDT0000: new disk, initializing [11212.961092] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11212.979313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11215.624870] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11223.354162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11223.441039] Lustre: 273709:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11223.499966] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11223.506403] Lustre: Skipped 1 previous similar message [11223.580463] Lustre: lustre-MDT0001: new disk, initializing [11223.646454] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11223.674839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11223.683704] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11226.366845] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11231.422441] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11234.109957] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11238.413288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11238.603454] Lustre: lustre-OST0000: new disk, initializing [11238.606272] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11238.662058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11242.269368] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11246.625191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11246.633638] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11246.706867] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11247.661435] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11251.448096] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [11256.663254] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [11262.417112] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [11308.004381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11308.004924] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11308.007243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11308.017230] Lustre: Skipped 1 previous similar message [11313.361043] Lustre: server umount lustre-OST0000 complete [11316.192515] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11316.195966] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11316.214320] Lustre: Skipped 1 previous similar message [11316.220855] Lustre: Skipped 3 previous similar messages [11322.135788] Lustre: server umount lustre-MDT0000 complete [11324.947324] LustreError: 273701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773618395 with bad export cookie 2071434418370749025 [11324.953631] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11324.955189] LustreError: 273701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11325.176590] Lustre: server umount lustre-MDT0001 complete [11331.866404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11334.986201] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11338.799919] Lustre: server umount MGS complete [11346.695264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11347.060291] LustreError: 279131:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11347.081384] LustreError: 279144:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [11347.140784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11350.130634] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11352.546756] LustreError: 279132:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11356.646256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11359.677985] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11363.375799] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11365.703220] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11370.017634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11374.409413] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11378.228838] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11403.234759] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11403.236815] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11403.250127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11403.255910] Lustre: Skipped 1 previous similar message [11403.259966] Lustre: Skipped 3 previous similar messages [11406.884258] Lustre: server umount lustre-OST0000 complete [11413.473067] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11423.199680] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11423.363651] Lustre: server umount lustre-MDT0000 complete [11423.713902] LustreError: 279844:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11423.727361] LustreError: 279844:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11425.409983] LustreError: 279852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773618495 with bad export cookie 2071434418370750754 [11425.414722] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11425.429958] LustreError: 279852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11425.592609] Lustre: server umount lustre-MDT0001 complete [11432.215586] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [11433.895961] Key type lgssc unregistered [11434.116868] LNet: 282724:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11434.121443] LNetError: 282724:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11434.132345] LNet: Removed LNI 192.168.201.125@tcp [11434.698044] Key type .llcrypt unregistered [11434.699507] Key type ._llcrypt unregistered [11459.494370] Key type ._llcrypt registered [11459.496789] Key type .llcrypt registered [11459.572882] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [11469.959138] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [11470.520903] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11470.543041] alg: No test for adler32 (adler32-zlib) [11471.468948] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [11471.628123] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [11473.271340] Key type lgssc registered [11473.920152] Lustre: Echo OBD driver; http://www.lustre.org/ [11494.324503] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11494.335719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11495.490757] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11495.519241] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11495.596116] Lustre: lustre-MDT0000: new disk, initializing [11495.647911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11495.659811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11497.892712] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11504.155936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11504.237473] Lustre: 286622:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11504.280371] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11504.284492] Lustre: Skipped 1 previous similar message [11504.339421] Lustre: lustre-MDT0001: new disk, initializing [11504.408294] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11504.443052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11504.453589] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11506.586676] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11511.255502] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11513.596557] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11517.720588] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11517.946150] Lustre: lustre-OST0000: new disk, initializing [11517.950887] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11518.027399] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11522.166408] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11523.619059] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11523.626878] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11523.707303] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11526.898137] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11532.413651] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [11543.189969] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [11553.262040] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [11563.540840] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [11610.596766] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11610.599812] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11610.603590] Lustre: Skipped 1 previous similar message [11610.604123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11610.613725] Lustre: Skipped 1 previous similar message [11612.647944] Lustre: server umount lustre-OST0000 complete [11615.713840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11615.715229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11615.727667] Lustre: Skipped 1 previous similar message [11615.733117] Lustre: Skipped 1 previous similar message [11620.735759] Lustre: server umount lustre-MDT0000 complete [11622.551985] LustreError: 286614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773618692 with bad export cookie 13061422501044443447 [11622.557796] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11622.560572] LustreError: 286614:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11622.752640] Lustre: server umount lustre-MDT0001 complete [11627.060919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11629.284336] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11631.982570] Lustre: server umount MGS complete [11637.841116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11638.121247] LustreError: 292391:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11638.133460] LustreError: 292391:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [11638.138451] LustreError: 292404:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [11638.179237] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11640.120979] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11643.364841] LustreError: 292392:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11644.116853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11644.303341] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11646.150393] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11648.768763] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11650.576858] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11654.438635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11654.568884] LustreError: 293984:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11654.660736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11658.176870] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11661.395307] Lustre: DEBUG MARKER: oleg125-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11692.513623] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11692.518072] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11692.525866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11693.539607] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11694.769526] Lustre: server umount lustre-OST0000 complete [11698.657197] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11698.659277] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11698.664946] Lustre: Skipped 2 previous similar messages [11698.668428] Lustre: Skipped 2 previous similar messages [11703.159208] Lustre: server umount lustre-MDT0000 complete [11703.776603] LustreError: 293105:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11703.785713] LustreError: 293105:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [11705.007731] LustreError: 292373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773618775 with bad export cookie 13061422501044445386 [11705.010458] LustreError: MGC192.168.201.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11705.014846] LustreError: 292373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11705.194522] Lustre: server umount lustre-MDT0001 complete [11710.187332] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing unload_modules_local [11711.662963] Key type lgssc unregistered [11711.831120] LNet: 295983:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11711.835679] LNetError: 295983:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11711.846347] LNet: Removed LNI 192.168.201.125@tcp [11712.274788] Key type .llcrypt unregistered [11712.277052] Key type ._llcrypt unregistered [11721.751427] Key type ._llcrypt registered [11721.752928] Key type .llcrypt registered [11721.815865] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_hostid [11729.713701] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [11730.176573] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11730.229051] alg: No test for adler32 (adler32-zlib) [11731.152833] Lustre: Lustre: Build Version: 2.17.51_2_gc98ff3f [11731.302408] LNet: Added LNI 192.168.201.125@tcp [8/256/0/180] [11732.911408] Key type lgssc registered [11733.448854] Lustre: Echo OBD driver; http://www.lustre.org/ [11746.204962] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing load_modules_local [11753.860727] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11753.867811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11754.989602] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11755.006942] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11755.061657] Lustre: lustre-MDT0000: new disk, initializing [11755.102830] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11755.115206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11756.912178] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11763.039317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11763.240067] Lustre: lustre-OST0000: new disk, initializing [11763.243534] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11763.255332] Lustre: Skipped 1 previous similar message [11763.302861] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11766.800905] Lustre: DEBUG MARKER: oleg125-server.virtnet: executing set_default_debug -1 all [11771.953054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [11771.962922] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [11771.991042] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400