[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 402752550 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002221] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004006] kvm-guest: setup PV IPIs [ 0.006000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006009] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007004] pid_max: default: 32768 minimum: 301 [ 0.008073] LSM: Security Framework initializing [ 0.009024] Yama: becoming mindful. [ 0.009721] SELinux: Initializing. [ 0.010037] *** VALIDATE selinux *** [ 0.017367] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.021016] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.022080] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023060] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024051] *** VALIDATE tmpfs *** [ 0.025117] *** VALIDATE proc *** [ 0.025891] *** VALIDATE cgroup *** [ 0.026005] *** VALIDATE cgroup2 *** [ 0.027171] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.028096] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.029003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.030015] Spectre V2 : User space: Vulnerable [ 0.030833] Speculative Store Bypass: Vulnerable [ 0.033260] debug: unmapping init [mem 0xffffffffb4e59000-0xffffffffb4e60fff] [ 0.035132] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.037299] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.038010] ... version: 2 [ 0.038879] ... bit width: 48 [ 0.039006] ... generic registers: 4 [ 0.039796] ... value mask: 0000ffffffffffff [ 0.040006] ... max period: 00007fffffffffff [ 0.041005] ... fixed-purpose events: 3 [ 0.042006] ... event mask: 000000070000000f [ 0.044105] rcu: Hierarchical SRCU implementation. [ 0.046064] smp: Bringing up secondary CPUs ... [ 0.047295] x86: Booting SMP configuration: [ 0.048014] .... node #0, CPUs: #1 #2 #3 [ 0.050717] smp: Brought up 1 node, 4 CPUs [ 0.051865] smpboot: Max logical packages: 1 [ 0.052008] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.235021] node 0 deferred pages initialised in 182ms [ 0.237181] devtmpfs: initialized [ 0.238154] x86/mm: Memory block size: 128MB [ 0.240278] gcov: version magic: 0x41383552 [ 0.241539] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.242079] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.243152] pinctrl core: initialized pinctrl subsystem [ 0.244096] [ 0.244445] ************************************************************* [ 0.245007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.246006] ** ** [ 0.247008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.248006] ** ** [ 0.249006] ** This means that this kernel is built to expose internal ** [ 0.250005] ** IOMMU data structures, which may compromise security on ** [ 0.251006] ** your system. ** [ 0.252010] ** ** [ 0.253006] ** If you see this message and you are not debugging the ** [ 0.254005] ** kernel, report this immediately to your vendor! ** [ 0.255006] ** ** [ 0.256005] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.257005] ************************************************************* [ 0.258447] NET: Registered protocol family 16 [ 0.259309] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.260027] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.261025] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.262260] cpuidle: using governor menu [ 0.263029] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.265396] PCI: Using configuration type 1 for base access [ 0.267089] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.272116] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.273012] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.274101] cryptd: max_cpu_qlen set to 1000 [ 0.275171] ACPI: Added _OSI(Module Device) [ 0.276022] ACPI: Added _OSI(Processor Device) [ 0.277000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.278010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.281081] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.285688] ACPI: Interpreter enabled [ 0.287050] ACPI: PM: (supports S0 S3 S4 S5) [ 0.288008] ACPI: Using IOAPIC for interrupt routing [ 0.289052] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.291257] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.297807] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.299017] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.300007] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.302036] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.305920] acpiphp: Slot [2] registered [ 0.307079] acpiphp: Slot [5] registered [ 0.307928] acpiphp: Slot [6] registered [ 0.308058] acpiphp: Slot [7] registered [ 0.309007] acpiphp: Slot [8] registered [ 0.309920] acpiphp: Slot [9] registered [ 0.311074] acpiphp: Slot [10] registered [ 0.312009] acpiphp: Slot [3] registered [ 0.312969] acpiphp: Slot [4] registered [ 0.314046] acpiphp: Slot [11] registered [ 0.314978] acpiphp: Slot [12] registered [ 0.316044] acpiphp: Slot [13] registered [ 0.317031] acpiphp: Slot [14] registered [ 0.317946] acpiphp: Slot [15] registered [ 0.318042] acpiphp: Slot [16] registered [ 0.318948] acpiphp: Slot [17] registered [ 0.320042] acpiphp: Slot [18] registered [ 0.320913] acpiphp: Slot [19] registered [ 0.322056] acpiphp: Slot [20] registered [ 0.322893] acpiphp: Slot [21] registered [ 0.323041] acpiphp: Slot [22] registered [ 0.323981] acpiphp: Slot [23] registered [ 0.325042] acpiphp: Slot [24] registered [ 0.325854] acpiphp: Slot [25] registered [ 0.327042] acpiphp: Slot [26] registered [ 0.327863] acpiphp: Slot [27] registered [ 0.329042] acpiphp: Slot [28] registered [ 0.329982] acpiphp: Slot [29] registered [ 0.330049] acpiphp: Slot [30] registered [ 0.331022] acpiphp: Slot [31] registered [ 0.331950] PCI host bridge to bus 0000:00 [ 0.334009] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.335007] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.337008] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.338007] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.340008] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.342010] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.343090] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.344702] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.346864] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.353008] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.356056] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.358007] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.359009] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.361008] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.363216] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.365519] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.367019] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.368467] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.372007] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.381008] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.385008] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.389159] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.393007] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.396007] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.406007] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.411990] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.414860] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.419007] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.429007] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.435793] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.438949] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.441766] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.453744] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.458904] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.462007] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.465992] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.479009] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.485680] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.489822] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.493889] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.507010] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.513236] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.518007] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.520866] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.534011] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.544184] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.547400] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.550303] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.553192] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.554107] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.558697] iommu: Default domain type: Passthrough [ 0.561267] SCSI subsystem initialized [ 0.562075] ACPI: bus type USB registered [ 0.562926] usbcore: registered new interface driver usbfs [ 0.564029] usbcore: registered new interface driver hub [ 0.565028] usbcore: registered new device driver usb [ 0.566084] pps_core: LinuxPPS API ver. 1 registered [ 0.568006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.569015] PTP clock support registered [ 0.571102] EDAC MC: Ver: 3.0.0 [ 0.572113] PCI: Using ACPI for IRQ routing [ 0.573577] NetLabel: Initializing [ 0.574006] NetLabel: domain hash size = 128 [ 0.574985] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.575038] NetLabel: unlabeled traffic allowed by default [ 0.576110] vgaarb: loaded [ 0.577196] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.578006] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.582306] clocksource: Switched to clocksource kvm-clock [ 0.671989] VFS: Disk quotas dquot_6.6.0 [ 0.672958] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.674500] *** VALIDATE ramfs *** [ 0.675290] *** VALIDATE hugetlbfs *** [ 0.676241] pnp: PnP ACPI init [ 0.677799] pnp: PnP ACPI: found 6 devices [ 0.691261] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.693202] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.694511] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.695829] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.697369] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.698792] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.700466] NET: Registered protocol family 2 [ 0.702068] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.705289] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.707407] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.711193] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.713189] TCP: Hash tables configured (established 65536 bind 65536) [ 0.714829] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.716793] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.718568] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.720300] NET: Registered protocol family 1 [ 0.721925] RPC: Registered named UNIX socket transport module. [ 0.723287] RPC: Registered udp transport module. [ 0.724310] RPC: Registered tcp transport module. [ 0.725298] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.726698] NET: Registered protocol family 44 [ 0.727729] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.729096] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.730465] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.731839] PCI: CLS 0 bytes, default 64 [ 0.732829] Unpacking initramfs... [ 1.951994] debug: unmapping init [mem 0xffff9d087cc54000-0xffff9d087ffbffff] [ 1.956648] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.958123] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 1.959903] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.373803] Initialise system trusted keyrings [ 2.374930] Key type blacklist registered [ 2.376151] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.382090] zbud: loaded [ 2.383900] *** VALIDATE nfs *** [ 2.384678] *** VALIDATE nfs4 *** [ 2.385663] pstore: using deflate compression [ 2.387432] Platform Keyring initialized [ 2.457234] NET: Registered protocol family 38 [ 2.458523] Key type asymmetric registered [ 2.459427] Asymmetric key parser 'x509' registered [ 2.460523] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.462444] io scheduler mq-deadline registered [ 2.463491] io scheduler kyber registered [ 2.464634] io scheduler bfq registered [ 2.465745] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.467566] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.469325] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.471075] ACPI: Power Button [PWRF] [ 2.474299] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.478705] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.485706] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.489801] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.498684] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.523924] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.549826] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.552729] Non-volatile memory driver v1.3 [ 2.553750] Linux agpgart interface v0.103 [ 2.573189] virtio_blk virtio1: [vda] 133944 512-byte logical blocks (68.6 MB/65.4 MiB) [ 2.574920] vda: detected capacity change from 0 to 68579328 [ 2.584208] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.585941] vdb: detected capacity change from 0 to 1073741824 [ 2.596947] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.599928] vdc: detected capacity change from 0 to 2621440000 [ 2.611772] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.614784] vdd: detected capacity change from 0 to 2621440000 [ 2.626132] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.629132] vde: detected capacity change from 0 to 4294967296 [ 2.639347] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.641112] vdf: detected capacity change from 0 to 4294967296 [ 2.644272] libphy: Fixed MDIO Bus: probed [ 2.647142] usbcore: registered new interface driver usbserial_generic [ 2.648514] usbserial: USB Serial support registered for generic [ 2.649787] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.652458] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.653542] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.655081] mousedev: PS/2 mouse device common for all mice [ 2.656801] rtc_cmos 00:05: RTC can wake from S4 [ 2.658205] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.660983] rtc_cmos 00:05: registered as rtc0 [ 2.661321] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.662170] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.664601] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.665634] intel_pstate: CPU model not supported [ 2.669069] hid: raw HID events driver (C) Jiri Kosina [ 2.670214] usbcore: registered new interface driver usbhid [ 2.671360] usbhid: USB HID core driver [ 2.672252] drop_monitor: Initializing network drop monitor service [ 2.673685] Initializing XFRM netlink socket [ 2.674909] NET: Registered protocol family 10 [ 2.676517] Segment Routing with IPv6 [ 2.677390] NET: Registered protocol family 17 [ 2.678519] mpls_gso: MPLS GSO support [ 2.683227] RAS: Correctable Errors collector initialized. [ 2.684527] AVX version of gcm_enc/dec engaged. [ 2.685550] AES CTR mode by8 optimization enabled [ 2.724239] sched_clock: Marking stable (2724224185, 0)->(3362752787, -638528602) [ 2.726067] registered taskstats version 1 [ 2.727164] Loading compiled-in X.509 certificates [ 2.728388] zswap: loaded using pool lzo/zbud [ 2.740959] Key type big_key registered [ 2.747485] Key type encrypted registered [ 2.748347] ima: No TPM chip found, activating TPM-bypass! [ 2.749500] ima: Allocated hash algorithm: sha1 [ 2.750466] ima: No architecture policies found [ 2.751503] evm: Initialising EVM extended attributes: [ 2.752598] evm: security.selinux [ 2.753291] evm: security.ima [ 2.753888] evm: security.capability [ 2.754650] evm: HMAC attrs: 0x1 [ 2.756042] rtc_cmos 00:05: setting system clock to 2026-04-01 12:59:58 UTC (1775048398) [ 2.759792] debug: unmapping init [mem 0xffffffffb5e03000-0xffffffffb5ffffff] [ 2.761588] debug: unmapping init [mem 0xffffffffb4b82000-0xffffffffb4e58fff] [ 2.770062] Write protecting the kernel read-only data: 28672k [ 2.771858] debug: unmapping init [mem 0xffffffffb3203000-0xffffffffb33fffff] [ 2.773415] debug: unmapping init [mem 0xffffffffb3b14000-0xffffffffb3bfffff] [ 2.800079] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 2.809703] systemd[1]: Detected virtualization kvm. [ 2.810897] systemd[1]: Detected architecture x86-64. [ 2.812107] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 2.836811] systemd[1]: No hostname configured. [ 2.837888] systemd[1]: Set hostname to . [ 2.839130] random: systemd: uninitialized urandom read (16 bytes read) [ 2.840610] systemd[1]: Initializing machine ID from random generator. [ 2.873859] random: ln: uninitialized urandom read (6 bytes read) [ 2.933577] random: systemd: uninitialized urandom read (16 bytes read) [ 2.935294] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 2.940693] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ 2.945847] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.337547] device-mapper: uevent: version 1.0.3 [ 3.338912] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 3.776784] random: fast init done [ 3.783232] virtio_net virtio0 ens2: renamed from eth0 [ 3.810864] scsi host0: ata_piix [ 3.824597] scsi host1: ata_piix [ 3.825603] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 3.827095] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 7.890092] dracut-initqueue[589]: RTNETLINK answers: File exists [ 8.258155] random: crng init done [ 8.259128] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 8.534181] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 9.308441] printk: systemd: 26 output lines suppressed due to ratelimiting [ 9.468641] SELinux: Disabled at runtime. [ 9.519117] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 9.528597] systemd[1]: Detected virtualization kvm. [ 9.529726] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 9.839989] systemd[1]: initrd-switch-root.service: Succeeded. [ 9.841961] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 9.844651] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 9.846868] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 9.848685] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 9.853807] systemd[1]: Starting Journal Service... Starting Journal Service... [ 9.856460] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on initctl Compatibility Named Pipe. [ 9.919209] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 10.174316] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 10.385508] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 10.428200] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 10.511176] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 10.524396] EDAC sbridge: Ver: 1.1.2 [ 11.540765] Key type dns_resolver registered [ 11.809938] NFS: Registering the id_resolver key type [ 11.811270] Key type id_resolver registered [ 11.812256] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. [ OK ] Started Crash recovery kernel arming. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg325-server login: [ 30.788411] libcfs: loading out-of-tree module taints kernel. [ 30.800955] Key type ._llcrypt registered [ 30.801910] Key type .llcrypt registered [ 30.839436] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 37.833053] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 38.518764] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 38.524839] alg: No test for adler32 (adler32-zlib) [ 39.502580] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 39.784621] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 41.399161] Key type lgssc registered [ 41.958498] Lustre: Echo OBD driver; http://www.lustre.org/ [ 47.653155] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 59.655892] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 63.819735] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 63.827404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 64.917106] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 64.931367] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 64.973163] Lustre: lustre-MDT0000: new disk, initializing [ 64.997193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 65.003404] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 66.254238] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 71.093609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 71.127828] Lustre: 6467:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 71.139287] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 71.141012] Lustre: Skipped 1 previous similar message [ 71.175893] Lustre: lustre-MDT0001: new disk, initializing [ 71.193956] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 71.201111] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 71.203486] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 72.373620] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 74.507124] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 77.436883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 77.525319] Lustre: lustre-OST0000: new disk, initializing [ 77.527099] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 77.543607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 79.259170] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 81.642779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 81.645593] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 81.661271] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 84.256323] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 84.292840] Lustre: lustre-OST0001: new disk, initializing [ 84.294329] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 84.309979] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 86.010594] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 90.090725] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 90.093471] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 90.107813] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 91.629862] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 95.087124] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 101.892862] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing check_logdir /tmp/testlogs/ [ 103.190610] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing yml_node [ 104.500358] Lustre: DEBUG MARKER: Client: 2.17.51.74 [ 105.268346] Lustre: DEBUG MARKER: MDS: 2.17.51.74 [ 106.086814] Lustre: DEBUG MARKER: OSS: 2.17.51.74 [ 106.599824] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Apr 1 09:01:42 EDT 2026 [ 111.953365] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 112.462233] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 125.919902] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 125.920345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 125.924142] Lustre: Skipped 3 previous similar messages [ 125.928154] Lustre: Skipped 3 previous similar messages [ 130.198960] Lustre: server umount lustre-MDT0000 complete [ 131.306601] LustreError: 6462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048527 with bad export cookie 13814590236254641107 [ 131.307785] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 131.310824] LustreError: 6462:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 136.159457] LustreError: 6477:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 136.159634] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 136.159881] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 136.166597] LustreError: 6477:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 136.171989] Lustre: Skipped 1 previous similar message [ 137.444344] Lustre: server umount lustre-MDT0001 complete [ 144.764516] Lustre: server umount lustre-OST0000 complete [ 160.223107] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 160.275895] Lustre: server umount lustre-OST0001 complete [ 162.241249] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 164.879862] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 175.829679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 175.920521] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 175.932159] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 175.974462] Lustre: lustre-MDT0000: new disk, initializing [ 175.997763] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 176.003748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 177.222020] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 181.043815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 181.085299] Lustre: 14949:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 181.190068] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 182.353933] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 185.229959] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 186.380341] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 188.518855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 188.638230] Lustre: lustre-OST0000: new disk, initializing [ 188.639720] Lustre: Skipped 1 previous similar message [ 188.641588] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 188.643351] Lustre: Skipped 2 previous similar messages [ 188.670690] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 188.672593] Lustre: Skipped 1 previous similar message [ 190.314166] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 190.588885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 190.591141] Lustre: Skipped 1 previous similar message [ 190.592834] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 190.634820] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 193.202334] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 194.331621] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 194.388980] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 195.492852] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 195.549920] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 201.185134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 201.186255] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 201.187216] LustreError: Skipped 1 previous similar message [ 201.187996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 201.191276] Lustre: Skipped 1 previous similar message [ 201.192415] Lustre: Skipped 1 previous similar message [ 202.185947] Lustre: server umount lustre-OST0000 complete [ 206.304687] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 206.305308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 206.308048] Lustre: Skipped 1 previous similar message [ 206.309591] Lustre: Skipped 1 previous similar message [ 209.435815] Lustre: server umount lustre-MDT0000 complete [ 210.536166] LustreError: 16689:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048606 with bad export cookie 13814590236254642626 [ 210.538878] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 210.539254] LustreError: 16689:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 210.648240] Lustre: server umount lustre-MDT0001 complete [ 212.089543] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:03:28 (1775048608) [ 214.105959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 214.262684] LustreError: 18802:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 214.285684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 215.375016] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 217.861149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 219.129870] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 220.685845] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 221.734078] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 223.814435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.930190] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 227.789037] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 233.637591] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 238.560071] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 238.562753] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 238.567077] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 238.569014] Lustre: Skipped 1 previous similar message [ 242.768934] Lustre: server umount lustre-MDT0000 complete [ 243.872217] LustreError: 18782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048639 with bad export cookie 13814590236254643508 [ 243.874780] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 243.877347] LustreError: 18782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 243.981938] Lustre: server umount lustre-MDT0001 complete [ 255.313353] Lustre: server umount lustre-OST0000 complete [ 261.417745] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 264.664472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 264.805684] LustreError: 22824:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 264.823913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 264.825626] Lustre: Skipped 2 previous similar messages [ 266.028296] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 268.591088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 269.783102] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 270.627441] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 272.564559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 274.264848] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 276.727758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 276.779358] Lustre: lustre-OST0001: new disk, initializing [ 276.781227] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 278.397177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 278.400218] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 278.400637] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 278.426294] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 282.502593] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 289.734784] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 289.736745] Lustre: Skipped 1 previous similar message [ 294.367680] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 294.369984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 294.373368] Lustre: Skipped 2 previous similar messages [ 294.374868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 294.376399] Lustre: Skipped 3 previous similar messages [ 299.798431] Lustre: server umount lustre-MDT0000 complete [ 300.000542] LustreError: 23537:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 300.005033] LustreError: 23537:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 300.889082] LustreError: 23547:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048696 with bad export cookie 13814590236254644355 [ 300.892085] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 300.892268] LustreError: 23547:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 300.992975] Lustre: server umount lustre-MDT0001 complete [ 311.295576] Lustre: server umount lustre-OST0000 complete [ 322.642571] Lustre: server umount lustre-OST0001 complete [ 327.458888] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 330.407118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 330.533839] LustreError: 28119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.550471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 330.552149] Lustre: Skipped 3 previous similar messages [ 331.639235] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 334.074525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 335.310512] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 336.147738] Lustre: 29235:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 338.042761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 339.807898] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 341.218696] LustreError: 29590:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 342.372133] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 344.011231] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 346.806275] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 347.794582] Lustre: 31082:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 354.783789] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 354.785903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 354.788921] Lustre: Skipped 3 previous similar messages [ 354.790611] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 354.792335] Lustre: Skipped 3 previous similar messages [ 360.282527] Lustre: server umount lustre-MDT0000 complete [ 361.406059] LustreError: 28101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048757 with bad export cookie 13814590236254645650 [ 361.409201] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 361.409400] LustreError: 28101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 361.512957] Lustre: server umount lustre-MDT0001 complete [ 373.183724] Lustre: server umount lustre-OST0000 complete [ 383.515204] Lustre: server umount lustre-OST0001 complete [ 386.192589] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:06:22 (1775048782) [ 394.794591] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 397.775431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 397.905826] LustreError: 33861:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 399.028017] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 401.660959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 402.889730] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 403.745575] Lustre: 34976:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 406.254066] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 408.193653] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 410.780588] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 412.659746] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 415.579161] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 416.643677] Lustre: 36823:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 427.490179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 427.494472] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 427.498657] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 427.506146] Lustre: Skipped 6 previous similar messages [ 427.511777] Lustre: Skipped 6 previous similar messages [ 428.974934] Lustre: server umount lustre-MDT0000 complete [ 430.459277] LustreError: 33843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048826 with bad export cookie 13814590236254646756 [ 430.465163] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 430.465789] LustreError: 33843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 430.593373] Lustre: server umount lustre-MDT0001 complete [ 441.793732] Lustre: server umount lustre-OST0000 complete [ 453.211379] Lustre: server umount lustre-OST0001 complete [ 455.750799] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:07:31 (1775048851) [ 460.532657] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 463.541154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 463.675226] LustreError: 38946:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 463.682974] LustreError: 38946:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 463.698636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 463.700968] Lustre: Skipped 7 previous similar messages [ 464.893181] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 467.429466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 468.688983] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 469.571053] Lustre: 40059:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 471.805798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 473.733407] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 476.864258] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 478.865245] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 481.911959] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 488.298516] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 493.023907] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 493.026089] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 493.029670] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 493.031278] Lustre: Skipped 2 previous similar messages [ 495.392983] Lustre: server umount lustre-MDT0000 complete [ 496.608545] LustreError: 41912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048892 with bad export cookie 13814590236254647862 [ 496.612055] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 496.613605] LustreError: 41912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 502.759746] Lustre: server umount lustre-MDT0001 complete [ 505.366748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 506.463137] LustreError: 42771:0:(import.c:337:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 506.465755] LustreError: 42771:0:(import.c:361:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff9d07c5de6680 x1861273188075264/t0(0) o250->MGC192.168.203.125@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1775048902 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_03.0' uid:0 gid:0 projid:4294967295 [ 506.471438] LustreError: 42771:0:(import.c:371:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 506.783523] LustreError: 3621:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9d08f823c700 x1861273188076672/t0(0) o250->MGC192.168.203.125@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 508.128061] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 510.777307] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 512.120071] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 513.772275] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 526.309977] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 530.776358] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 16 sec [ 531.438213] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 531.440185] Lustre: Skipped 1 previous similar message [ 531.921902] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 532.491441] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 536.545558] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 539.859893] Lustre: server umount lustre-MDT0000 complete [ 541.001019] LustreError: 38926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775048936 with bad export cookie 13814590236254649003 [ 541.004165] LustreError: 38926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 541.004476] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 541.111754] Lustre: server umount lustre-MDT0001 complete [ 552.579594] Lustre: server umount lustre-OST0000 complete [ 564.120326] Lustre: server umount lustre-OST0001 complete [ 567.248583] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:09:23 (1775048963) [ 572.394494] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 576.494891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 576.663861] LustreError: 46295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 576.675084] LustreError: 46295:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 577.926057] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 581.257914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 582.697568] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 583.616141] Lustre: 47411:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 583.620166] Lustre: 47411:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 586.384035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 588.733579] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 592.444160] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 594.607231] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 598.007465] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 605.000514] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 612.320182] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 615.581831] Lustre: server umount lustre-MDT0000 complete [ 616.885042] LustreError: 49321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049012 with bad export cookie 13814590236254649822 [ 616.888295] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 616.889899] LustreError: 49321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 617.009449] Lustre: server umount lustre-MDT0001 complete [ 627.646657] Lustre: server umount lustre-OST0000 complete [ 639.092876] Lustre: server umount lustre-OST0001 complete [ 643.647594] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:10:39 (1775049039) [ 644.603642] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 645.242994] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:10:41 (1775049041) [ 649.114221] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 651.808449] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 661.984390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 662.132033] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 662.182879] Lustre: lustre-MDT0000: new disk, initializing [ 662.215022] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 663.666911] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 667.653367] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 670.523077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 672.215420] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 672.231742] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 673.285923] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 677.273876] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 678.368570] Lustre: DEBUG MARKER: create test files [ 692.706698] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 692.706980] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 692.720599] Lustre: Skipped 13 previous similar messages [ 692.724363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 692.728912] Lustre: Skipped 18 previous similar messages [ 695.011108] Lustre: server umount lustre-OST0000 complete [ 696.571241] Lustre: server umount lustre-MDT0000 complete [ 701.457463] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 702.965503] Key type lgssc unregistered [ 703.139999] LNet: 54792:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 703.146226] LNetError: 54792:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 703.157556] LNet: Removed LNI 192.168.203.125@tcp [ 703.634163] Key type .llcrypt unregistered [ 703.636994] Key type ._llcrypt unregistered [ 716.386206] Key type ._llcrypt registered [ 716.387909] Key type .llcrypt registered [ 716.446970] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 716.923066] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 716.938385] alg: No test for adler32 (adler32-zlib) [ 717.827309] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 717.933059] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 719.519221] Key type lgssc registered [ 720.044033] Lustre: Echo OBD driver; http://www.lustre.org/ [ 723.065294] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 723.077279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 724.273113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 726.003017] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 728.256861] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 731.538314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 731.694401] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 734.386436] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 734.764219] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 734.806497] Lustre: 57370:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 734.814259] Lustre: 57370:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 734.820002] Lustre: 57370:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 734.824994] Lustre: 57370:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 734.830363] Lustre: 57370:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 734.836460] Lustre: 57370:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 736.859947] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 740.086965] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 742.139063] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 744.928266] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 744.928447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 744.934477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 750.048839] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 750.226598] Lustre: server umount lustre-OST0000 complete [ 751.536071] Lustre: server umount lustre-MDT0000 complete [ 755.244134] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 756.205905] Key type lgssc unregistered [ 756.332440] LNet: 58852:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 756.334735] LNetError: 58852:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 756.342412] LNet: Removed LNI 192.168.203.125@tcp [ 756.630769] Key type .llcrypt unregistered [ 756.632972] Key type ._llcrypt unregistered [ 767.231912] Key type ._llcrypt registered [ 767.232958] Key type .llcrypt registered [ 767.268100] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 767.600520] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 767.636564] alg: No test for adler32 (adler32-zlib) [ 768.476370] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 768.551660] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 770.127143] Key type lgssc registered [ 770.442589] Lustre: Echo OBD driver; http://www.lustre.org/ [ 772.743973] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 772.750532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 773.893231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 775.130513] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 776.758373] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 779.708628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 779.906253] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 782.385875] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 784.036785] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 784.059218] Lustre: 61532:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 784.062339] Lustre: 61532:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 784.064584] Lustre: 61532:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 784.067075] Lustre: 61532:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 784.069884] Lustre: 61532:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 784.072355] Lustre: 61532:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 784.511549] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 787.817775] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 794.592563] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 794.592767] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 794.599503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 796.303672] Lustre: server umount lustre-OST0000 complete [ 797.831863] Lustre: server umount lustre-MDT0000 complete [ 801.335196] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 802.425011] Key type lgssc unregistered [ 802.570393] LNet: 62960:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 802.573804] LNetError: 62960:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 802.584299] LNet: Removed LNI 192.168.203.125@tcp [ 802.864022] Key type .llcrypt unregistered [ 802.865028] Key type ._llcrypt unregistered [ 808.870312] Key type ._llcrypt registered [ 808.872909] Key type .llcrypt registered [ 808.914791] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 815.401802] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 815.798855] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 815.806723] alg: No test for adler32 (adler32-zlib) [ 816.699721] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 816.809668] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 818.415209] Key type lgssc registered [ 818.992951] Lustre: Echo OBD driver; http://www.lustre.org/ [ 832.172137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 832.177593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 833.274932] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 833.314652] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 833.364140] Lustre: lustre-MDT0000: new disk, initializing [ 833.391483] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 833.399458] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 835.069005] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 839.779561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 839.842279] Lustre: 66051:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 839.883038] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 839.886976] Lustre: Skipped 1 previous similar message [ 839.943193] Lustre: lustre-MDT0001: new disk, initializing [ 839.994886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 840.015545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 840.022476] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 841.228709] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 844.705740] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 846.067783] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 848.571338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 848.752309] Lustre: lustre-OST0000: new disk, initializing [ 848.756798] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 848.808096] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 850.242152] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 850.248525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 850.304168] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 851.128950] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 854.846938] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 856.458841] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 856.524646] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 858.055688] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 858.178580] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 860.642597] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 860.643248] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 860.643937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 860.645416] LustreError: Skipped 1 previous similar message [ 860.652905] Lustre: Skipped 1 previous similar message [ 865.241189] Lustre: server umount lustre-OST0000 complete [ 870.881169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 870.881617] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 870.882814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 870.882818] Lustre: Skipped 1 previous similar message [ 870.884405] Lustre: Skipped 1 previous similar message [ 872.700206] Lustre: server umount lustre-MDT0000 complete [ 874.543899] LustreError: 66830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049270 with bad export cookie 9315949163757490855 [ 874.552454] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 874.553043] LustreError: 66830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 874.695793] Lustre: server umount lustre-MDT0001 complete [ 878.577171] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:14:34 (1775049274) [ 883.610442] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 901.371136] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 904.922429] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 915.729694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 915.814824] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 915.824408] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 915.858426] Lustre: lustre-MDT0000: new disk, initializing [ 915.884870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 915.890574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 917.070822] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 920.902361] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 920.937756] Lustre: 73606:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 920.956834] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 920.958372] Lustre: Skipped 1 previous similar message [ 920.998834] Lustre: lustre-MDT0001: new disk, initializing [ 921.028394] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 921.039392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 921.042778] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 922.225411] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 925.144716] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 926.282430] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 928.261340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 929.399049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 929.401873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 929.428880] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 930.016283] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 932.955887] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 934.223707] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 934.282716] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 935.427663] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 935.508117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 940.001123] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 940.001184] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 940.003382] LustreError: Skipped 1 previous similar message [ 940.007572] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 940.009721] Lustre: Skipped 2 previous similar messages [ 942.411792] Lustre: server umount lustre-OST0000 complete [ 945.120675] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 945.121459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 945.124993] Lustre: Skipped 2 previous similar messages [ 945.129319] Lustre: Skipped 1 previous similar message [ 949.721360] Lustre: server umount lustre-MDT0000 complete [ 951.190973] LustreError: 73597:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049346 with bad export cookie 9315949163757491723 [ 951.197437] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 951.198117] LustreError: 73597:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 951.339425] Lustre: server umount lustre-MDT0001 complete [ 954.947172] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:15:51 (1775049351) [ 957.610120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 957.835386] LustreError: 77610:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 957.844381] LustreError: 77610:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 957.866216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 957.869796] Lustre: Skipped 1 previous similar message [ 959.304101] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 962.713596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 964.360316] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 966.428707] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 967.998364] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 971.067936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 973.594843] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 976.021286] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 977.694889] Lustre: 79943:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting 8646ed9c-d078-45a7-9078-0f1f9ff137ab at adminstrative request [ 978.408340] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 978.417526] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 988.650128] Lustre: MGS: Client 8646ed9c-d078-45a7-9078-0f1f9ff137ab (at 0@lo) reconnecting [ 988.655876] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8148e6680b5cd96f to 0x8148e6680b5cdbfa [ 988.665588] Lustre: MGC192.168.203.125@tcp: Connection restored to 0@lo (at 0@lo) [ 1010.515950] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1010.606526] Lustre: lustre-OST0001: new disk, initializing [ 1010.609459] Lustre: Skipped 1 previous similar message [ 1010.612639] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1010.616780] Lustre: Skipped 1 previous similar message [ 1010.658588] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1010.662187] Lustre: Skipped 2 previous similar messages [ 1013.173122] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1017.150680] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1018.518600] Lustre: server umount lustre-OST0001 complete [ 1018.890764] LustreError: 79205:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1023.456077] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1023.459309] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1023.465295] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1023.971221] LustreError: 79205:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1023.978056] LustreError: 79205:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1029.095656] LustreError: 79205:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1029.101414] LustreError: 79205:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1034.207128] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1034.212095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1034.212813] LustreError: 79956:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1034.214781] Lustre: Skipped 4 previous similar messages [ 1034.222939] LustreError: 79956:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1034.294704] Lustre: server umount lustre-OST0000 complete [ 1039.331822] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1039.341911] Lustre: Skipped 2 previous similar messages [ 1042.254032] Lustre: server umount lustre-MDT0000 complete [ 1044.166620] LustreError: 79225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049439 with bad export cookie 9315949163757493242 [ 1044.173218] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1044.177069] LustreError: 79225:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1044.324636] Lustre: server umount lustre-MDT0001 complete [ 1048.033865] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:17:24 (1775049444) [ 1048.545216] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1049.328908] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:17:25 (1775049445) [ 1050.100232] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1050.759019] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:17:26 (1775049446) [ 1051.223474] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1051.827535] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:17:27 (1775049447) [ 1052.483481] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1053.179138] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:17:29 (1775049449) [ 1053.781263] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1055.908809] loop: module loaded [ 1055.912969] loop0: detected capacity change from 0 to 4294967296 [ 1055.923521] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1056.406517] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1057.362844] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1058.632869] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1066.897028] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1066.909273] LustreError: 84325:0:(obd_config.c:843:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1066.911272] LustreError: 84325:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1066.913242] LustreError: 84325:0:(tgt_mount.c:2513:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1066.915397] LustreError: 84325:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1072.499789] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 1075.114585] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1087.802198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1087.888922] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1087.899498] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1087.934098] Lustre: lustre-MDT0000: new disk, initializing [ 1087.957619] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1087.963270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1089.299780] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1093.747220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1093.785332] Lustre: 87150:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1093.886821] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1095.251168] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1098.313131] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1099.702948] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1102.669450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1102.858084] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1102.903293] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1104.756674] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1108.215209] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1109.611715] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1109.728190] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1111.543463] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1111.663969] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1113.059235] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1113.061100] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1113.061434] LustreError: Skipped 1 previous similar message [ 1113.068412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1113.070558] Lustre: Skipped 3 previous similar messages [ 1118.667202] Lustre: server umount lustre-OST0000 complete [ 1125.996811] Lustre: server umount lustre-MDT0000 complete [ 1127.787896] LustreError: 87142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049523 with bad export cookie 9315949163757493851 [ 1127.794910] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1127.796066] LustreError: 87142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1127.953477] Lustre: server umount lustre-MDT0001 complete [ 1131.014684] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:18:46 (1775049526) [ 1133.358935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1133.501931] LustreError: 91150:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1134.784098] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1138.139585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1139.696847] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1142.033724] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1143.682996] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1146.556987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1149.207816] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1151.592618] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1153.491904] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1159.751929] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 6 sec [ 1161.229632] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1161.335705] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1164.541613] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1164.643904] Lustre: lustre-OST0001: new disk, initializing [ 1164.646976] Lustre: Skipped 2 previous similar messages [ 1164.650712] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1164.654493] Lustre: Skipped 3 previous similar messages [ 1164.704770] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1164.708393] Lustre: Skipped 5 previous similar messages [ 1166.740060] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1170.321492] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1172.133920] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1174.550587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1174.556864] Lustre: Skipped 2 previous similar messages [ 1174.561005] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1174.604246] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1175.327448] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1176.988834] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1177.108665] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1187.778509] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1193.808331] Lustre: Failing over lustre-MDT0000 [ 1194.217108] Lustre: server umount lustre-MDT0000 complete [ 1194.981618] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1194.984560] LustreError: 91145:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1194.989035] Lustre: Skipped 6 previous similar messages [ 1194.999203] LustreError: 91145:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1195.002064] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1210.225205] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1210.228167] LDISKFS-fs (dm-0): recovery complete [ 1210.234492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1210.297661] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1210.336719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (not set up) [ 1210.340751] Lustre: Skipped 7 previous similar messages [ 1210.442595] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1211.930057] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1214.278415] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1214.279167] Lustre: 92998:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1215.463181] Lustre: 91151:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1215.469503] Lustre: 91151:0:(ldlm_lib.c:2081:extend_recovery_timer()) Skipped 1 previous similar message [ 1215.478119] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1215.506895] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1215.823079] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1216.137637] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1216.142046] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 1 previous similar message [ 1216.455133] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1216.460488] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 1 previous similar message [ 1216.466866] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1216.474175] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) Skipped 7 previous similar messages [ 1217.417431] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1217.421848] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 3 previous similar messages [ 1217.735063] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1217.737332] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 3 previous similar messages [ 1218.680780] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1218.684029] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) Skipped 6 previous similar messages [ 1219.623676] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1219.626781] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 6 previous similar messages [ 1219.943070] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1219.945428] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 6 previous similar messages [ 1222.767695] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1222.771058] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) Skipped 12 previous similar messages [ 1223.712780] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1223.715552] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 12 previous similar messages [ 1224.031133] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1224.039958] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 12 previous similar messages [ 1230.911727] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1230.915768] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) Skipped 25 previous similar messages [ 1231.859085] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1231.866408] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 25 previous similar messages [ 1232.183132] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1232.187954] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 25 previous similar messages [ 1247.216503] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1247.224977] Lustre: 96890:0:(ldlm_lib.c:2081:extend_recovery_timer()) Skipped 51 previous similar messages [ 1247.864481] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1247.870510] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 50 previous similar messages [ 1248.191130] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1248.195848] LustreError: 96890:0:(ldlm_lib.c:2694:replay_request_or_update()) Skipped 50 previous similar messages [ 1274.152509] Lustre: 96890:0:(ldlm_lib.c:2390:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1274.156811] Lustre: 96890:0:(ldlm_lib.c:2400:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1274.169136] Lustre: 96890:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 2af6d1a1-bd30-4690-8264-45ed47502829@192.168.203.25@tcp [ 1274.185882] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1274.200713] Lustre: 96890:0:(ldlm_lib.c:1910:abort_req_replay_queue()) @@@ aborted: req@0000000067f02f6f x1861273997350144/t0(8589934784) o35->2af6d1a1-bd30-4690-8264-45ed47502829@192.168.203.25@tcp:135/0 lens 392/0 e 0 to 0 dl 1775049680 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1274.222035] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1274.230653] Lustre: lustre-MDT0000: Denying connection for new client 2af6d1a1-bd30-4690-8264-45ed47502829 (at 192.168.203.25@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1274.238474] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1274.277096] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1274.281398] Lustre: Skipped 2 previous similar messages [ 1274.286955] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1274.320243] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1274.324059] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1285.093879] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1285.104300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1285.121129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1298.399079] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1298.432538] Lustre: server umount lustre-OST0000 complete [ 1302.499995] LustreError: 93000:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1302.506243] LustreError: 93000:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 15 previous similar messages [ 1305.750286] Lustre: server umount lustre-OST0001 complete [ 1313.199889] Lustre: server umount lustre-MDT0000 complete [ 1314.495265] LustreError: 91176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049710 with bad export cookie 9315949163757545035 [ 1314.497423] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1314.500065] LustreError: 91176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1314.611903] Lustre: server umount lustre-MDT0001 complete [ 1317.425236] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:21:53 (1775049713) [ 1319.747941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1319.750399] Lustre: *** cfs_fail_loc=197, val=0*** [ 1335.199737] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1340.319373] LustreError: 98850:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.125@tcp: failed processing log, type 4: rc = -110 [ 1345.446527] Lustre: *** cfs_fail_loc=197, val=0*** [ 1345.448679] Lustre: Skipped 65 previous similar messages [ 1365.919585] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1365.922813] Lustre: Skipped 1 previous similar message [ 1367.852488] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1369.743925] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1370.443415] Lustre: server umount lustre-OST0000 complete [ 1373.327943] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:22:49 (1775049769) [ 1387.135664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1387.294344] LustreError: 100354:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1388.721824] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1391.906304] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1393.995510] LustreError: 100386:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1393.999670] LustreError: 100386:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1394.003542] LustreError: 100386:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -5 [ 1394.127754] Lustre: server umount lustre-MDT0000 complete [ 1405.559669] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 1409.060557] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1421.097439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1421.178931] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1421.189713] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1421.227912] Lustre: lustre-MDT0000: new disk, initializing [ 1421.255076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1422.545056] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1426.702182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1426.743301] Lustre: 104401:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1426.852182] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1428.244661] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1431.425706] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1432.689916] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1435.033448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1437.110474] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1438.755201] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1440.424948] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1441.890279] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1441.967460] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1443.296899] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1443.368262] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1448.930720] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1448.930730] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1448.935828] Lustre: Skipped 5 previous similar messages [ 1448.939571] LustreError: Skipped 3 previous similar messages [ 1448.942074] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1448.945855] Lustre: Skipped 14 previous similar messages [ 1450.127060] Lustre: server umount lustre-OST0000 complete [ 1457.498952] Lustre: server umount lustre-MDT0000 complete [ 1458.736904] LustreError: 104394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049854 with bad export cookie 9315949163757555766 [ 1458.739064] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1458.740151] LustreError: 104394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1458.844421] Lustre: server umount lustre-MDT0001 complete [ 1461.598499] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:24:17 (1775049857) [ 1464.535240] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 1465.490501] Key type lgssc unregistered [ 1465.614371] LNet: 108460:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1465.616665] LNetError: 108460:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1465.625275] LNet: Removed LNI 192.168.203.125@tcp [ 1465.898092] Key type .llcrypt unregistered [ 1465.899181] Key type ._llcrypt unregistered [ 1471.409369] Key type ._llcrypt registered [ 1471.410537] Key type .llcrypt registered [ 1471.446433] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 1477.444844] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1477.744139] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1477.763545] alg: No test for adler32 (adler32-zlib) [ 1478.617957] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 1478.700049] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 1480.271160] Key type lgssc registered [ 1480.614265] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1493.713304] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1493.718445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1494.808171] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1494.819454] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1494.850597] Lustre: lustre-MDT0000: new disk, initializing [ 1494.872517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1494.878800] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1496.178701] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1500.414223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1500.523809] Lustre: lustre-OST0000: new disk, initializing [ 1500.525553] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1500.527200] Lustre: Skipped 1 previous similar message [ 1500.550467] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1502.512204] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1502.651318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1502.654350] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1502.663156] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1506.740762] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1506.796655] Lustre: lustre-OST0001: new disk, initializing [ 1506.798758] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1506.823717] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1508.750017] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1510.206054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1510.209845] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1510.218332] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1515.488633] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1515.489141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1515.492997] Lustre: Skipped 1 previous similar message [ 1518.885798] Lustre: server umount lustre-MDT0000 complete [ 1521.152543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1522.752626] LustreError: 111756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775049918 with bad export cookie 13884993984445247700 [ 1522.753351] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1522.756399] LustreError: 111756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1522.790220] Lustre: server umount lustre-OST0000 complete [ 1524.128418] Lustre: server umount lustre-OST0001 complete [ 1527.319453] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 09:25:23 (1775049923) [ 1530.992985] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1539.596207] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:25:35 (1775049935) [ 1554.553421] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 1558.393275] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1571.388259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1571.474371] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1571.486172] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1571.522370] Lustre: lustre-MDT0000: new disk, initializing [ 1571.547079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1571.554337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1572.920735] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1577.157988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1577.195280] Lustre: 120070:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1577.217844] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1577.220431] Lustre: Skipped 1 previous similar message [ 1577.256878] Lustre: lustre-MDT0001: new disk, initializing [ 1577.297665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1577.309986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1577.313366] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1578.569284] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1581.561583] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1582.724616] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1584.989333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1586.960375] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1590.002702] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1591.253801] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1591.286307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1591.289745] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1591.313901] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1592.337668] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1593.515404] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1593.581064] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1596.385351] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1596.386158] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1596.388317] LustreError: Skipped 1 previous similar message [ 1596.393537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1596.397336] Lustre: Skipped 2 previous similar messages [ 1600.266869] Lustre: server umount lustre-OST0000 complete [ 1601.504577] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1601.505146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1601.508297] Lustre: Skipped 2 previous similar messages [ 1601.511660] Lustre: Skipped 1 previous similar message [ 1606.624084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1606.625797] Lustre: Skipped 1 previous similar message [ 1607.575238] Lustre: server umount lustre-MDT0000 complete [ 1608.770357] LustreError: 120065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050004 with bad export cookie 13884993984445248911 [ 1608.773977] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1608.774167] LustreError: 120065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1608.884276] Lustre: server umount lustre-MDT0001 complete [ 1611.569884] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:26:47 (1775050007) [ 1613.598924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1613.761727] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1613.764261] Lustre: Skipped 1 previous similar message [ 1615.027889] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1617.659649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1619.139993] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1620.917910] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1622.092560] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1624.236132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1626.139235] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1627.895981] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1645.377516] Lustre: *** cfs_fail_loc=159, val=0*** [ 1645.379506] Lustre: Skipped 1 previous similar message [ 1661.780674] Lustre: lustre-MDT0000: Client 38eaa86f-2120-41e0-8a41-0bd4f57bcab8 (at 192.168.203.25@tcp) reconnecting [ 1662.290633] Lustre: *** cfs_fail_loc=159, val=0*** [ 1662.292734] Lustre: Skipped 4 previous similar messages [ 1677.653432] Lustre: lustre-MDT0000: Client 38eaa86f-2120-41e0-8a41-0bd4f57bcab8 (at 192.168.203.25@tcp) reconnecting [ 1694.175834] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1694.179621] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1694.184081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1695.201230] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1699.602464] Lustre: server umount lustre-OST0000 complete [ 1704.927815] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1704.930827] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1704.935501] Lustre: Skipped 1 previous similar message [ 1704.937524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1704.939529] Lustre: Skipped 2 previous similar messages [ 1715.167123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1715.232924] Lustre: server umount lustre-MDT0000 complete [ 1715.679861] LustreError: 126435:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1715.685911] LustreError: 126435:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1716.439263] LustreError: 124064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050112 with bad export cookie 13884993984445249793 [ 1716.442852] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1716.443081] LustreError: 124064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1716.558138] Lustre: server umount lustre-MDT0001 complete [ 1720.258837] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 1721.284822] Key type lgssc unregistered [ 1721.417317] LNet: 127533:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1721.419995] LNetError: 127533:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1721.435327] LNet: Removed LNI 192.168.203.125@tcp [ 1721.731058] Key type .llcrypt unregistered [ 1721.732532] Key type ._llcrypt unregistered [ 1731.142400] Key type ._llcrypt registered [ 1731.144261] Key type .llcrypt registered [ 1731.186453] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1731.594547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1731.607331] alg: No test for adler32 (adler32-zlib) [ 1732.458705] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 1732.542081] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 1734.127148] Key type lgssc registered [ 1734.504312] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1737.021388] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1737.028168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1738.176858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1739.419106] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1742.193472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1742.314748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1743.555748] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1745.299074] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1746.527252] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1748.779733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1748.895165] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1750.880241] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1752.750731] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1763.238132] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1772.871764] Lustre: *** cfs_fail_loc=159, val=0*** [ 1789.265712] Lustre: lustre-MDT0000: Client 72aea697-d38c-4c77-b1c8-56974738484f (at 192.168.203.25@tcp) reconnecting [ 1790.003661] Lustre: *** cfs_fail_loc=159, val=0*** [ 1806.164590] Lustre: lustre-MDT0001: Client 72aea697-d38c-4c77-b1c8-56974738484f (at 192.168.203.25@tcp) reconnecting [ 1806.715532] Lustre: *** cfs_fail_loc=159, val=0*** [ 1806.717375] Lustre: Skipped 3 previous similar messages [ 1823.055966] Lustre: lustre-MDT0001: Client 72aea697-d38c-4c77-b1c8-56974738484f (at 192.168.203.25@tcp) reconnecting [ 1837.577725] Lustre: 131649:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1838.430675] Lustre: *** cfs_fail_loc=159, val=0*** [ 1838.432827] Lustre: Skipped 3 previous similar messages [ 1854.804592] Lustre: lustre-MDT0000: Client 375651d1-94a9-4c58-95b3-29037b09529a (at 192.168.203.25@tcp) reconnecting [ 1855.382296] Lustre: *** cfs_fail_loc=159, val=0*** [ 1855.384331] Lustre: Skipped 12 previous similar messages [ 1871.702258] Lustre: lustre-MDT0000: Client 375651d1-94a9-4c58-95b3-29037b09529a (at 192.168.203.25@tcp) reconnecting [ 1872.195297] Lustre: 131936:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1875.937157] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1875.937672] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1875.938300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1875.940726] Lustre: Skipped 1 previous similar message [ 1875.942767] LustreError: Skipped 1 previous similar message [ 1879.506752] Lustre: server umount lustre-OST0000 complete [ 1881.057676] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1881.058732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1881.061788] Lustre: Skipped 1 previous similar message [ 1881.063420] Lustre: Skipped 1 previous similar message [ 1886.176076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1886.179196] Lustre: Skipped 1 previous similar message [ 1891.296071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1891.298691] Lustre: Skipped 1 previous similar message [ 1895.391120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1895.392110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1895.397306] Lustre: Skipped 1 previous similar message [ 1895.455214] Lustre: server umount lustre-MDT0000 complete [ 1896.668247] LustreError: 130519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050292 with bad export cookie 10913565270277084645 [ 1896.671418] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1896.672579] LustreError: 130519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1896.781345] Lustre: server umount lustre-MDT0001 complete [ 1900.260360] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 1901.188535] Key type lgssc unregistered [ 1901.307337] LNet: 132972:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1901.309686] LNetError: 132972:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1901.316280] LNet: Removed LNI 192.168.203.125@tcp [ 1901.578523] Key type .llcrypt unregistered [ 1901.579992] Key type ._llcrypt unregistered [ 1910.199580] Key type ._llcrypt registered [ 1910.200840] Key type .llcrypt registered [ 1910.239242] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1910.644183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1910.655282] alg: No test for adler32 (adler32-zlib) [ 1911.505185] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 1911.587191] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 1913.168458] Key type lgssc registered [ 1913.525573] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1916.074584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1916.081331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1917.234401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1918.499493] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1921.404923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1921.524680] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1922.796637] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1924.504212] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1925.689666] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1927.945714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1928.064174] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1929.950809] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1931.713045] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1932.199259] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1941.419048] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1944.449816] Lustre: 136614:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1945.039901] Lustre: 136664:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1945.505976] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 1946.591076] Lustre: 136813:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 1946.882693] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 1946.885477] Lustre: Skipped 1 previous similar message [ 1947.171217] Lustre: 136911:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1951.713656] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1951.714066] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1951.721753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1951.723868] Lustre: Skipped 1 previous similar message [ 1954.192620] Lustre: server umount lustre-OST0000 complete [ 1956.833649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1956.838434] Lustre: Skipped 1 previous similar message [ 1956.840435] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1956.842079] Lustre: Skipped 1 previous similar message [ 1961.494804] Lustre: server umount lustre-MDT0000 complete [ 1961.952485] LustreError: 135070:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1961.959974] LustreError: 135070:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1962.659148] LustreError: 136715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050358 with bad export cookie 18149190536834720901 [ 1962.661533] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1962.663569] LustreError: 136715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1962.766590] Lustre: server umount lustre-MDT0001 complete [ 1966.028280] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 1967.002634] Key type lgssc unregistered [ 1967.129418] LNet: 137949:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1967.132119] LNetError: 137949:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1967.141328] LNet: Removed LNI 192.168.203.125@tcp [ 1967.413158] Key type .llcrypt unregistered [ 1967.415039] Key type ._llcrypt unregistered [ 1976.056778] Key type ._llcrypt registered [ 1976.058074] Key type .llcrypt registered [ 1976.090366] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 1976.470345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1976.499325] alg: No test for adler32 (adler32-zlib) [ 1977.346047] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 1977.431538] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 1979.007152] Key type lgssc registered [ 1979.351106] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1981.797785] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1981.803737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1982.948920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1984.185438] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1986.963258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1987.090860] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1988.346274] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1990.016224] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1991.092682] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1993.197701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1993.308053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1995.001755] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 1996.389393] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 1996.600098] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2007.653522] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2013.964559] Lustre: *** cfs_fail_loc=159, val=0*** [ 2013.966440] Lustre: Skipped 1 previous similar message [ 2029.394854] Lustre: lustre-MDT0000: Client 62cc4011-b9c6-446b-91c9-bfb5ee3e077a (at 192.168.203.25@tcp) reconnecting [ 2047.967837] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2047.970408] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2047.974588] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2048.992802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2048.992881] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2048.994144] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2048.995098] Lustre: Skipped 1 previous similar message [ 2051.468311] Lustre: server umount lustre-OST0000 complete [ 2053.599688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2053.602906] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2053.608992] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2058.835986] Lustre: server umount lustre-MDT0000 complete [ 2059.231921] LustreError: 139324:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2059.237604] LustreError: 139324:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2060.131338] LustreError: 139305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050455 with bad export cookie 3558716948934827942 [ 2060.134743] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2060.136110] LustreError: 139305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2060.253912] Lustre: server umount lustre-MDT0001 complete [ 2063.756096] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 2064.845709] Key type lgssc unregistered [ 2064.973404] LNet: 142669:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2064.976094] LNetError: 142669:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2064.983406] LNet: Removed LNI 192.168.203.125@tcp [ 2065.288701] Key type .llcrypt unregistered [ 2065.289845] Key type ._llcrypt unregistered [ 2074.405241] Key type ._llcrypt registered [ 2074.406448] Key type .llcrypt registered [ 2074.449930] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2074.852746] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2074.905039] alg: No test for adler32 (adler32-zlib) [ 2075.758819] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 2075.846032] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 2077.439099] Key type lgssc registered [ 2077.803748] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2080.408154] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2080.415665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2081.574049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2082.881106] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2085.757271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2085.894086] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2087.233178] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2089.122723] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2090.452147] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2092.756469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2092.881400] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2094.796977] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2095.973297] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2096.588796] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2107.237888] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2120.951943] Lustre: lustre-OST0000: evicting b1c599eb-dd29-413d-b9a2-5dbaa9274566 (at 192.168.203.25@tcp) by administrative request [ 2120.955509] Lustre: Skipped 1 previous similar message [ 2147.296689] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2147.298663] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2147.302830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2148.321124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2148.321800] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2148.322072] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2148.323056] Lustre: Skipped 1 previous similar message [ 2151.118111] Lustre: server umount lustre-OST0000 complete [ 2152.415778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2152.418048] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2152.421988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2158.484777] Lustre: server umount lustre-MDT0000 complete [ 2158.559855] LustreError: 144045:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2158.566497] LustreError: 144045:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2159.684056] LustreError: 144767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050555 with bad export cookie 4057727978389897608 [ 2159.686621] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2159.688376] LustreError: 144767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2159.801182] Lustre: server umount lustre-MDT0001 complete [ 2163.342860] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 2164.281606] Key type lgssc unregistered [ 2164.411474] LNet: 147777:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2164.415381] LNetError: 147777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2164.423311] LNet: Removed LNI 192.168.203.125@tcp [ 2164.712525] Key type .llcrypt unregistered [ 2164.713606] Key type ._llcrypt unregistered [ 2187.934400] Key type ._llcrypt registered [ 2187.935580] Key type .llcrypt registered [ 2187.968738] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2188.355877] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2188.362966] alg: No test for adler32 (adler32-zlib) [ 2189.206084] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 2189.278965] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 2190.855111] Key type lgssc registered [ 2191.183210] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2193.564911] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2193.570683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2194.727879] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2195.944318] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2198.534145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2198.645980] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2199.760741] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2201.347391] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2202.385123] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2204.459688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2204.561301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2205.608962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2206.349923] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2207.947379] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2208.677083] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2213.858411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2213.858754] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2213.859765] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2213.863085] Lustre: Skipped 1 previous similar message [ 2217.628193] Lustre: server umount lustre-OST0000 complete [ 2218.976286] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2218.976821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2218.980710] Lustre: Skipped 1 previous similar message [ 2218.983495] Lustre: Skipped 1 previous similar message [ 2224.096226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2224.098839] Lustre: Skipped 2 previous similar messages [ 2229.215989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2229.217924] Lustre: Skipped 1 previous similar message [ 2233.311137] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2233.313477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2233.317651] Lustre: Skipped 1 previous similar message [ 2233.372508] Lustre: server umount lustre-MDT0000 complete [ 2234.675362] LustreError: 150201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050630 with bad export cookie 15940499355703522499 [ 2234.679128] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2234.682447] LustreError: 150201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2234.797698] Lustre: server umount lustre-MDT0001 complete [ 2238.358468] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 2239.374656] Key type lgssc unregistered [ 2239.498391] LNet: 153405:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2239.501062] LNetError: 153405:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2239.511293] LNet: Removed LNI 192.168.203.125@tcp [ 2239.799184] Key type .llcrypt unregistered [ 2239.802066] Key type ._llcrypt unregistered [ 2248.906345] Key type ._llcrypt registered [ 2248.907639] Key type .llcrypt registered [ 2248.947765] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2249.531252] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2249.548300] alg: No test for adler32 (adler32-zlib) [ 2250.394898] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 2250.477101] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 2252.055129] Key type lgssc registered [ 2252.480220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2257.488811] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:37:33 (1775050653) [ 2262.765691] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 2265.904302] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2277.495642] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2277.503140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.587303] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2278.597668] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2278.634321] Lustre: lustre-MDT0000: new disk, initializing [ 2278.657132] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2278.663520] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2280.084561] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2284.402372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2284.444239] Lustre: 157601:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2284.480612] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2284.482886] Lustre: Skipped 1 previous similar message [ 2284.518614] Lustre: lustre-MDT0001: new disk, initializing [ 2284.551516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2284.563557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2284.568150] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2285.850699] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2288.869780] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2290.010885] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2292.219779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2292.321982] Lustre: lustre-OST0000: new disk, initializing [ 2292.324370] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2292.353314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2292.719914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2292.722868] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2292.760355] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2294.173459] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2297.203626] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2302.946165] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2302.946285] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2302.949834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2302.953638] Lustre: Skipped 1 previous similar message [ 2302.956899] Lustre: Skipped 1 previous similar message [ 2306.384680] Lustre: server umount lustre-OST0000 complete [ 2308.065409] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2308.066194] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2308.069333] Lustre: Skipped 1 previous similar message [ 2313.185830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2313.188144] Lustre: Skipped 2 previous similar messages [ 2313.765059] Lustre: server umount lustre-MDT0000 complete [ 2315.055574] LustreError: 157594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775050710 with bad export cookie 1885630810478336454 [ 2315.060399] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2315.060833] LustreError: 157594:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2315.192238] Lustre: server umount lustre-MDT0001 complete [ 2318.660196] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 2319.666435] Key type lgssc unregistered [ 2319.791332] LNet: 161119:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2319.793520] LNetError: 161119:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2319.801275] LNet: Removed LNI 192.168.203.125@tcp [ 2320.075849] Key type .llcrypt unregistered [ 2320.077040] Key type ._llcrypt unregistered [ 2329.043103] Key type ._llcrypt registered [ 2329.044817] Key type .llcrypt registered [ 2329.089325] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2329.510957] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2329.524358] alg: No test for adler32 (adler32-zlib) [ 2330.385331] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 2330.477572] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 2332.071163] Key type lgssc registered [ 2332.495635] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2335.053573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2335.060276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2336.225354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2337.566827] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2340.404127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2340.523469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2341.779913] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2343.544494] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2344.720451] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2346.990989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2347.125582] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2348.970348] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2350.714362] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2360.424075] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2363.585885] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2363.595702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2363.602612] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2363.606355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2363.611150] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2363.975278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.342372] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2364.345035] Lustre: Skipped 1 previous similar message [ 2364.350986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2364.354696] Lustre: Skipped 1 previous similar message [ 2364.356212] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2364.358890] Lustre: Skipped 1 previous similar message [ 2364.360468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.365295] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2364.367598] Lustre: Skipped 1 previous similar message [ 2364.671542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.980621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.295547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.611859] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2365.613964] Lustre: Skipped 3 previous similar messages [ 2365.618914] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2365.622427] Lustre: Skipped 3 previous similar messages [ 2365.623833] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2365.626372] Lustre: Skipped 3 previous similar messages [ 2365.627715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.631874] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2365.633805] Lustre: Skipped 3 previous similar messages [ 2365.932549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.229837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.531305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.860925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.158848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.454283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.734856] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2367.738181] Lustre: Skipped 6 previous similar messages [ 2367.743338] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2367.748869] Lustre: Skipped 6 previous similar messages [ 2367.750629] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2367.752421] Lustre: Skipped 6 previous similar messages [ 2367.753766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.758447] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2367.761718] Lustre: Skipped 6 previous similar messages [ 2368.053178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.342160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.631474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.949971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.253032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.562604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.900622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.192787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.534283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.836716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.135802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.439328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.750503] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2371.752649] Lustre: Skipped 12 previous similar messages [ 2371.758731] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2371.762491] Lustre: Skipped 12 previous similar messages [ 2371.764420] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2371.767131] Lustre: Skipped 12 previous similar messages [ 2371.769133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.772625] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2371.774538] Lustre: Skipped 12 previous similar messages [ 2372.083919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.402176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.692423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.996248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.286903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.588286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.885555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.190316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.486322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.779245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.064163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.371106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.683272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.001113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.319302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.645736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.931727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.248427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.551389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.889401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.185742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.478520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.774660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.081202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.428517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.861361] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2379.865262] Lustre: Skipped 25 previous similar messages [ 2379.872930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2379.879144] Lustre: Skipped 25 previous similar messages [ 2379.881517] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2379.885576] Lustre: Skipped 25 previous similar messages [ 2379.888507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.895090] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2379.897815] Lustre: Skipped 25 previous similar messages [ 2380.298025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.699680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.156059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.555716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.870561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.181273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.518901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.914290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.271691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.643964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.975655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.320950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.667231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.960361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.266958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.567347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.863647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.235405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.548349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.885547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.290989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.614333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.901061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.206078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.504318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.820479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.134248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.468167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.775653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.103177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.401949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.697138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.998684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.307248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.604895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.898681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.189636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.511988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.791866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.071060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.365984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.682215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.018839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.417809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.728885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.027581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.320322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.632978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.000543] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2396.003471] Lustre: Skipped 48 previous similar messages [ 2396.009791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2396.014271] Lustre: Skipped 48 previous similar messages [ 2396.016284] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2396.020319] Lustre: Skipped 48 previous similar messages [ 2396.023117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.026757] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2396.029321] Lustre: Skipped 48 previous similar messages [ 2396.311462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.600989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.921548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.302433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.641669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.060598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.498281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.923681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.379512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.785336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.201804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.632699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.993879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.296343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.682639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.029327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.374732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.735992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.164701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.573925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.911076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.278538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.606411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.911752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.222285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.531316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.827103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.141224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.434173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.757664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.054789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.372224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.666632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.955361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.247778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.541606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.825727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.135360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.439118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.739813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.038268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.348071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.740419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.138310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.436498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.731305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.080700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.406490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.712736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.037421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.365721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.678086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.964238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.314099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.632236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.997749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.348992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.763392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.214270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.538423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.853232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.157264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.479234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.788289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.117921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.477259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.799547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.181690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.578575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.883408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.198445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.519362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.826372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.133813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.454083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.804874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.139922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.450224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.791843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.144164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.557763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.877163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.203879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.609563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.908068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.200917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.505916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.809323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.111980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.400727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.688868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.004789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.304953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.594169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.895736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.201999] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2428.205192] Lustre: Skipped 95 previous similar messages [ 2428.211558] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2428.215364] Lustre: Skipped 95 previous similar messages [ 2428.216815] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2428.219133] Lustre: Skipped 95 previous similar messages [ 2428.220690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.224476] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2428.226867] Lustre: Skipped 95 previous similar messages [ 2428.516956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.828286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.136833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.454990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.802992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.135689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.517299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.940660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.243557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.567821] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2431.571992] LustreError: 162513:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2431.573104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.907183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.242321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.584928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.926254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.247850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.584142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.025703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.385433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.766609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.106570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.485644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.794667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.118981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.424261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.745911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.066092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.356314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.649124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.939163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.258672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.603064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.003609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.454548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.899605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.236383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.564541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.889280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.194023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.498550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.817297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.131116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.428090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.736811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.038604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.369994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.750913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.152216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.469062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.776751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.114692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.451813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.789051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.105442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.423430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.756435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.071769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.374065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.690338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.042836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.344563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.699851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.046288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.417707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.763665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.155961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.500152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.839108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.184409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.537511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.881434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.187347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.527233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.855091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.169129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.468163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.761803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.092081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.406716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.715856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.032630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.347891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.624747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.964061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.325095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.690656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.008779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.334051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.652368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.974393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.334935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.671451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.007202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.422840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.866226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.250702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.617922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.957464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.299352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.642041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.956058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.285401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.607715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.964923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.282749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.578507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.907974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.219439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.533225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.845478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.177754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.497156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.818376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.127799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.434852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.734858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.039785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.340687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.651438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.943513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.236856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.531409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.856631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.159986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.459630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.773986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.069835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.358782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.659174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.957298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.272743] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2471.275931] LustreError: 162513:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2471.278190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.568861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.876157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.195779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.508854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.794987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.087240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.385661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.695632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.989860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.274143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.551220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.850719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.170734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.511606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.835717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.140845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.430120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.744216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.041569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.344881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.641406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.983501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.302559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.600271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.883786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.187753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.492324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.807426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.101195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.391936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.686726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.986398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.284792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.581131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.900714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.235936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.544189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.869363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.167540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.469867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.765468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.092167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.402874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.706706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.034779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.355610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.677443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.998142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.312861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.657995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.997404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.310266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.625259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.941123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.254936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.591465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.907191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.231436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.563100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.879846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.271106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.698744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.057173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.388479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.698601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.090902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.427160] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2492.429393] Lustre: Skipped 196 previous similar messages [ 2492.436738] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2492.440897] Lustre: Skipped 196 previous similar messages [ 2492.442495] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2492.444678] Lustre: Skipped 196 previous similar messages [ 2492.446149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.449385] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2492.451264] Lustre: Skipped 196 previous similar messages [ 2492.784238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.092581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.386251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.712794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.068202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.424026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.722162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.076935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.483903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.924454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.223336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.549823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.929877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.284962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.620939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.999711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.382260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.807468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.161287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.479058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.846390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.187200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.556887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.872924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.179250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.640349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.018203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.400455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.813296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.193226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.577710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.955804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.281779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.615938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.941992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.303829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.650559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.981679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.293843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.615010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.901923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.211733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.497582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.788932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.077182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.371747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.658905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.964894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.248249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.551703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.839102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.145131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.433760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.713644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.000765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.303524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.594458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.888452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.181845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.476500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.764840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.040759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.329130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.629548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.912406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.210579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.506630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.794284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.082250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.358518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.645234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.935485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.227147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.517695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.779138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.059983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.346037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.633151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.917347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.202834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.470736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.749034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.032902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.329627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.634467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.947226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.263188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.572378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.862805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.161179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.466129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.782710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.071565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.358349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.658144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.965598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.258901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.557753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.846212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.159331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.450943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.740575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.025086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.320054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.602282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.901718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.205377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.498977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.790967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.075508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.365331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.655434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.947706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.252380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.537846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.851034] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2528.854369] LustreError: 162513:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2528.856148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.159897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.474410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.776860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.097956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.404314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.727681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.047479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.375516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.685290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.006223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.323870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.654916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.969898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.281490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.590851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.914257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.207091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.514908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.820893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.118975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.403199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.697843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.997549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.307077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.602571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.887993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.182810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.485679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.779275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.082896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.365846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.666507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.952854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.235116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.525242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.830829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.114545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.397601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.683804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.979433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.269812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.550948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.853269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.150854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.446723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.732722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.013933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.328803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.615982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.929896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.222250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.538598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.836991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.138681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.426725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.720326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.008895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.294487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.579770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.903847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.194112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.488586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.787751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.100471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.392910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.696470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.978433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.275181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.567803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.865553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.171489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.486274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.777591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.063127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.354994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.652435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.947055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.227560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.504801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.802178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.083516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.376426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.663756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.954668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.227742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.512188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.798649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.104698] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2555.107957] LustreError: 162513:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2555.110296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.408715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.714567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.015235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.329158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.622337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.913243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.207824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.504388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.808908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.103453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.406816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.720958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.007057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.311468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.612438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.934827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.244420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.561798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.867308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.183572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.482205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.781512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.074655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.372317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.655520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.946645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.247529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.568951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.868161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.155735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.456326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.756273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.040304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.332755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.612520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.908013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.177675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.457504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.748904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.055819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.346812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.641827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.939367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.262456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.563336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.867891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.165242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.469210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.779427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.078541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.388424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.697122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.974168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.265807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.556261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.876306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.185244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.477953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.805521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.135953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.448537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.749700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.041101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.339657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.620719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.911754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.203917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.504863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.795606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.095247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.400848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.713280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.001328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.283623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.567951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.873814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.161725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.438461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.726228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.020442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.317799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.630601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.926105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.237954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.530296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.840398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.131879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.423143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.697756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.985874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.272943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.582808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.876317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.162736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.451055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.751339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.030748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.315987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.612523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.938260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.239390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.542671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.837173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.131977] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2586.136044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.136606] LustreError: 162513:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2586.426767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.705428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.980534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.266348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.542625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.831461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.120405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.409837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.690524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.967834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.246920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.548844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.834810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.123909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.404018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.685834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.991980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.283166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.572942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.866582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.154910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.437599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.734183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.030306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.311930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.599441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.898551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.195382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.482362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.763450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.045503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.342662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.632718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.926896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.214513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.530302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.847338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.177044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.492242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.801693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.098470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.403782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.699665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.000756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.294798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.582611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.861887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.166615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.451760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.754970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.048568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.334137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.612450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.886372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.178768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.457928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.747347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.039932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.330250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.629716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.960387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.262753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.619840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.940407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.234670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.526164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.804198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.122771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.415245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.701236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.987349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.293598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.572819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.876226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.160900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.457861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.735472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.021993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.301192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.578597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.880781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.175746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.486143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.811107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.122499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.430482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.726800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.048832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.344289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.634502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.927722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.225158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.518133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.796254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.085549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.392530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.673229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.969991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.270353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.586205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.892099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.188500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.475264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.785531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.078065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.379132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.704124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.042806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.354815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.671207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.986586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.312652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.636339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.927630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.223047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.532024] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2620.534593] Lustre: Skipped 423 previous similar messages [ 2620.540757] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2620.544437] Lustre: Skipped 423 previous similar messages [ 2620.545932] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2620.547892] Lustre: Skipped 423 previous similar messages [ 2620.549297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.553032] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2620.556955] Lustre: Skipped 423 previous similar messages [ 2620.867380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.220654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.538457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.893815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.216292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.520301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.827271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.139238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.444066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.748622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.050725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.365157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.667444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.962811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.245678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.548213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.840975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.142849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.448074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.758109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.044438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.347287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.651528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.960210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.252644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.543557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.832729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.158115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.467495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.782768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.094682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.412507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.705867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.994074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.296366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.613726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.921731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.232222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.550935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.905556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.222911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.539600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.874110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.218415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.526186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.831932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.151547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.482059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.786573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.093904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.402430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.725432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.054438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.364927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.671436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.983522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.285972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.591456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.903141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.227462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.514169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.820489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.112061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.424195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.719939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.014245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.300641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.599632] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2641.603757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.604399] LustreError: 162513:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2641.904754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.200292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.486301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.801730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.092182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.380781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.677107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.985041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.296186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.606220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.912893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.226684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.532874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.831120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.123579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.429502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.721090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.028723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.324511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.626567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.920496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.214302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.504521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.845123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.155444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.447224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.747436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.059555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.357189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.641200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.940914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.254316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.571057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.914549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.246109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.578093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.888491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.192065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.499819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.815591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.125798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.431732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.771817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.113015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.416893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.708580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.005770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.333018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.656721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.958216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.258912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.564475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.857931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.160935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.446387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.740428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.028819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.314246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.608704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.925786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.225049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.530380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.825454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.151469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.451409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.741460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.054406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.407468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.732668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.050853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.362059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.685947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.985383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.290312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.586833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.914063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.220593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.543683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.845131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.195353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.504452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.803364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.102349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.419788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.721228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.012134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.306081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.630905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.949725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.253912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.553343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.864595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.158238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.453569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.742117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.057233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.348115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.649403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.936791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.239183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.547097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.852233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.140712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.458874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.763564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.069453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.361398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.658486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.947805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.248321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.558316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.868443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.175646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.075941] LustreError: 164088:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2738.080210] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2738.084342] LustreError: 162513:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 2801.123178] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2801.125156] Lustre: Skipped 1 previous similar message [ 2805.568203] Lustre: server umount lustre-OST0000 complete [ 2811.361410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2811.363081] Lustre: Skipped 1 previous similar message [ 2813.137257] Lustre: server umount lustre-MDT0000 complete [ 2814.311040] LustreError: 216141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051210 with bad export cookie 17110086172035939285 [ 2814.314201] LustreError: 216141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2814.314746] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2814.442969] Lustre: server umount lustre-MDT0001 complete [ 2817.973545] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 2819.016893] Key type lgssc unregistered [ 2819.137270] LNet: 217164:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2819.139531] LNetError: 217164:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2819.146273] LNet: Removed LNI 192.168.203.125@tcp [ 2819.430209] Key type .llcrypt unregistered [ 2819.431316] Key type ._llcrypt unregistered [ 2828.401909] Key type ._llcrypt registered [ 2828.402976] Key type .llcrypt registered [ 2828.448774] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2828.890681] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2828.901562] alg: No test for adler32 (adler32-zlib) [ 2829.758316] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 2829.841803] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 2831.415110] Key type lgssc registered [ 2831.758512] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2834.313225] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2834.320155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2835.473282] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2836.696962] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2839.576098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2839.707116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2840.945953] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2842.709521] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2843.945114] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2846.120752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2846.177886] LustreError: 220129:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2846.184341] LustreError: 220129:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2846.224131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2848.043791] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2849.319790] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 2849.922374] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2852.759528] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2853.360587] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2853.367378] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2853.371310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.375865] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2853.375883] LustreError: 218558:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2854.369642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2854.371043] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2854.375726] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2854.381617] Lustre: Skipped 1 previous similar message [ 2856.085548] Lustre: server umount lustre-OST0000 complete [ 2873.455135] LustreError: 218558:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2883.988454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2884.104625] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2885.902719] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2887.682473] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2894.369266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.372936] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.373095] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2894.379741] Lustre: Skipped 1 previous similar message [ 2898.549073] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 10 sec [ 2899.664842] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9462070fc000.ost_server_uuid 50 [ 2900.159798] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9462070fc000.ost_server_uuid in FULL state after 0 sec [ 2904.545268] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2904.545358] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2904.549037] Lustre: Skipped 1 previous similar message [ 2904.549744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2904.554220] Lustre: Skipped 1 previous similar message [ 2907.219344] Lustre: server umount lustre-OST0000 complete [ 2909.665619] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2909.666565] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2909.671241] Lustre: Skipped 2 previous similar messages [ 2914.650338] Lustre: server umount lustre-MDT0000 complete [ 2914.784265] LustreError: 218539:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2914.790109] LustreError: 218539:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2915.890282] LustreError: 220929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051311 with bad export cookie 1277405379440912646 [ 2915.894849] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2916.018856] Lustre: server umount lustre-MDT0001 complete [ 2919.578646] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 2920.635172] Key type lgssc unregistered [ 2920.763818] LNet: 223204:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2920.768439] LNetError: 223204:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2920.778483] LNet: Removed LNI 192.168.203.125@tcp [ 2921.080452] Key type .llcrypt unregistered [ 2921.082356] Key type ._llcrypt unregistered [ 2932.370184] Key type ._llcrypt registered [ 2932.371595] Key type .llcrypt registered [ 2932.409591] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 2932.838879] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2932.847838] alg: No test for adler32 (adler32-zlib) [ 2933.699042] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 2933.789154] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 2935.375244] Key type lgssc registered [ 2935.784388] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2938.927055] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2938.932461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2940.092068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2941.387085] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2944.215532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2944.341642] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2945.698825] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2946.738485] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 2948.971494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2949.115520] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2950.119018] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 2951.302440] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2954.186292] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2954.249868] Lustre: lustre-OST0001: new disk, initializing [ 2954.251798] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2954.287497] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2956.192610] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 2956.745392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2956.748670] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2956.791866] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2961.588602] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2966.211466] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 2980.319775] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2980.321887] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2980.325592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2982.368592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2982.371295] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2982.376575] Lustre: Skipped 2 previous similar messages [ 2985.699647] Lustre: server umount lustre-MDT0000 complete [ 2986.925255] LustreError: 225607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051382 with bad export cookie 11687361002948311006 [ 2986.928966] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2986.929764] LustreError: 225607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2987.042025] Lustre: server umount lustre-MDT0001 complete [ 2998.531830] Lustre: server umount lustre-OST0000 complete [ 3010.012203] Lustre: server umount lustre-OST0001 complete [ 3017.187950] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3020.880531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3021.036433] LustreError: 230224:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3021.043049] LustreError: 230224:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3021.058455] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3022.358645] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3025.204104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3026.659674] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3027.650699] Lustre: 231329:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3029.905705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3030.026710] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3030.029333] Lustre: Skipped 1 previous similar message [ 3031.077968] LustreError: 231683:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3031.084348] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 3031.931673] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3034.792049] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3036.734489] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3039.895770] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3041.091754] Lustre: 233170:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3050.975903] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3050.980318] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3050.987760] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3050.989678] Lustre: Skipped 3 previous similar messages [ 3056.545547] Lustre: server umount mylustre-MDT0000 complete [ 3056.608113] LustreError: 230224:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3056.612823] LustreError: 230224:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3057.785761] LustreError: 233172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051453 with bad export cookie 11687361002948313771 [ 3057.789533] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3057.789811] LustreError: 233172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3057.905911] Lustre: server umount mylustre-MDT0001 complete [ 3068.928122] Lustre: server umount mylustre-OST0000 complete [ 3079.390217] Lustre: server umount mylustre-OST0001 complete [ 3086.011378] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3089.284151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3089.444957] LustreError: 235458:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3089.468669] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3089.471051] Lustre: Skipped 1 previous similar message [ 3090.779238] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3093.567777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3095.072628] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3096.033244] Lustre: 236567:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3098.209684] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3099.363255] LustreError: 236921:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3099.366576] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49793) [ 3100.167888] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3102.985186] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3104.107596] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 3104.900319] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3108.014678] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3109.210714] Lustre: 238408:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3119.584983] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3119.587227] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3119.589596] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3119.590759] Lustre: Skipped 5 previous similar messages [ 3119.592819] Lustre: Skipped 5 previous similar messages [ 3122.266349] Lustre: server umount tfs-MDT0000 complete [ 3123.500237] LustreError: 236185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051519 with bad export cookie 11687361002948315437 [ 3123.503256] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3123.504553] LustreError: 236185:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3123.617963] Lustre: server umount tfs-MDT0001 complete [ 3135.109623] Lustre: server umount tfs-OST0000 complete [ 3146.656164] Lustre: server umount tfs-OST0001 complete [ 3153.270607] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3156.800441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3156.949571] LustreError: 240639:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3156.972990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3156.975255] Lustre: Skipped 3 previous similar messages [ 3158.329925] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3161.241964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3162.703416] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3163.704465] Lustre: 241745:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3165.909773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3167.077865] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49825) [ 3167.838496] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3170.744766] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3171.883187] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 3172.639460] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3175.667376] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3176.822499] Lustre: 243588:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3184.027596] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 09:53:00 (1775051580) [ 3197.410636] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3197.412313] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3197.413129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3197.413134] Lustre: Skipped 2 previous similar messages [ 3197.414105] Lustre: Skipped 2 previous similar messages [ 3207.653129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3207.654907] Lustre: Skipped 9 previous similar messages [ 3210.207080] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3210.270581] Lustre: server umount lustre-MDT0000 complete [ 3211.425045] LustreError: 240615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051607 with bad export cookie 11687361002948317103 [ 3211.428571] LustreError: 240615:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3211.428723] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3212.767982] LustreError: 242120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3212.768879] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3212.772527] LustreError: 242120:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3212.779124] Lustre: Skipped 3 previous similar messages [ 3217.583065] Lustre: server umount lustre-MDT0001 complete [ 3233.247112] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3233.277362] Lustre: server umount lustre-OST0000 complete [ 3240.661328] Lustre: server umount lustre-OST0001 complete [ 3242.690412] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 3245.812568] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3257.735902] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3257.735902] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3257.735902] [ 3257.743495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3257.837773] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3257.848884] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3257.883747] Lustre: lustre-MDT0000: new disk, initializing [ 3257.907133] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3257.909240] Lustre: Skipped 3 previous similar messages [ 3257.915262] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3259.205856] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3263.356598] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3263.356598] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3263.356598] [ 3263.363298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3263.404108] Lustre: 246733:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3263.428174] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3263.430254] Lustre: Skipped 1 previous similar message [ 3263.476748] Lustre: lustre-MDT0001: new disk, initializing [ 3263.523434] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3263.527289] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3264.806910] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3268.966784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3269.068715] Lustre: lustre-OST0000: new disk, initializing [ 3269.070766] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3270.159934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3270.166037] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3270.212425] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3271.159840] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3275.219694] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3275.304487] Lustre: lustre-OST0001: new disk, initializing [ 3275.306392] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3277.065879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3277.070229] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3277.109729] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3277.269764] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3281.471791] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 09:54:37 (1775051677) [ 3284.950666] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 09:54:41 (1775051681) [ 3297.767243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3297.768244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3297.774083] Lustre: Skipped 1 previous similar message [ 3297.777202] Lustre: Skipped 4 previous similar messages [ 3302.460926] Lustre: server umount lustre-OST0000 complete [ 3302.883274] LustreError: 248265:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3302.887760] LustreError: 248265:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3304.415771] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3318.239168] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3318.312610] Lustre: server umount lustre-MDT0000 complete [ 3319.535065] LustreError: 246725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051715 with bad export cookie 11687361002948318657 [ 3319.538740] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3319.539444] LustreError: 246725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3325.685942] Lustre: server umount lustre-MDT0001 complete [ 3329.340031] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 3339.100979] Lustre: server umount lustre-OST0001 complete [ 3341.165338] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 3347.417489] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3359.442311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3359.549543] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3359.563314] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3359.608397] Lustre: lustre-MDT0000: new disk, initializing [ 3359.640449] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3361.000100] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3365.248657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3365.293819] Lustre: 253587:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3365.419719] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3366.664611] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3369.732606] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3370.890052] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3373.192653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3374.520444] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3374.554648] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3375.234723] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3378.345765] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3384.804947] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3384.805327] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3384.806602] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3384.806606] Lustre: Skipped 13 previous similar messages [ 3384.814640] Lustre: Skipped 5 previous similar messages [ 3386.836588] Lustre: server umount lustre-OST0000 complete [ 3394.219456] Lustre: server umount lustre-MDT0000 complete [ 3395.039937] LustreError: 253600:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3395.044368] LustreError: 253600:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 3395.439277] LustreError: 254366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051791 with bad export cookie 11687361002948320127 [ 3395.442302] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3395.444446] LustreError: 254366:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3395.560260] Lustre: server umount lustre-MDT0001 complete [ 3399.220813] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 3400.242907] Key type lgssc unregistered [ 3400.374352] LNet: 256957:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3400.377372] LNetError: 256957:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3400.387301] LNet: Removed LNI 192.168.203.125@tcp [ 3400.680116] Key type .llcrypt unregistered [ 3400.681480] Key type ._llcrypt unregistered [ 3410.380952] Key type ._llcrypt registered [ 3410.382188] Key type .llcrypt registered [ 3410.418531] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3410.856641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3410.870704] alg: No test for adler32 (adler32-zlib) [ 3411.727595] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 3411.804142] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 3413.383282] Key type lgssc registered [ 3413.858402] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3416.347232] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3416.355530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3417.513886] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3418.839767] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3421.659694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3421.782399] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3423.029709] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3424.746866] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3425.949751] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3428.170573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3428.286975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3430.150791] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3431.903458] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3432.229445] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3436.512693] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3436.515793] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3436.520526] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3438.928063] Lustre: server umount lustre-OST0000 complete [ 3441.633630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3441.635088] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3441.641528] Lustre: Skipped 1 previous similar message [ 3446.304103] Lustre: server umount lustre-MDT0000 complete [ 3446.752044] LustreError: 258379:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3446.756788] LustreError: 258379:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3447.545069] LustreError: 258359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051843 with bad export cookie 2138915312699522078 [ 3447.547427] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3447.549144] LustreError: 258359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3447.682959] Lustre: server umount lustre-MDT0001 complete [ 3451.248215] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 3452.319900] Key type lgssc unregistered [ 3452.451463] LNet: 261608:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3452.453869] LNetError: 261608:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3452.465432] LNet: Removed LNI 192.168.203.125@tcp [ 3452.794616] Key type .llcrypt unregistered [ 3452.795723] Key type ._llcrypt unregistered [ 3459.458637] Key type ._llcrypt registered [ 3459.459763] Key type .llcrypt registered [ 3459.500677] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3459.865667] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3459.897983] alg: No test for adler32 (adler32-zlib) [ 3460.763414] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 3460.859669] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 3462.439200] Key type lgssc registered [ 3462.823251] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3465.872647] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3465.879887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3481.311402] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3486.431225] LustreError: 262763:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.125@tcp: failed processing log, type 4: rc = -110 [ 3513.055177] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3514.984626] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3516.747311] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3517.520929] Lustre: server umount lustre-OST0000 complete [ 3523.055894] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 3524.075993] Key type lgssc unregistered [ 3524.207351] LNet: 264260:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3524.209786] LNetError: 264260:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3524.221264] LNet: Removed LNI 192.168.203.125@tcp [ 3524.501420] Key type .llcrypt unregistered [ 3524.502602] Key type ._llcrypt unregistered [ 3538.030659] Key type ._llcrypt registered [ 3538.031858] Key type .llcrypt registered [ 3538.071418] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3538.373394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3538.382117] alg: No test for adler32 (adler32-zlib) [ 3539.239677] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 3539.342183] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 3540.927184] Key type lgssc registered [ 3541.343063] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3545.743635] loop0: detected capacity change from 0 to 204800000 [ 3545.796926] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3545.800315] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3545.929380] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3546.289221] loop0: detected capacity change from 0 to 204800000 [ 3546.345570] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3546.348975] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3546.469385] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3546.811417] loop0: detected capacity change from 0 to 204800000 [ 3546.864931] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3546.868295] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3546.908535] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3547.287474] loop0: detected capacity change from 0 to 204800000 [ 3547.344721] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3547.743118] loop0: detected capacity change from 0 to 204800000 [ 3547.753592] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3549.109466] loop0: detected capacity change from 0 to 204800000 [ 3549.118212] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3550.401825] loop0: detected capacity change from 0 to 204800000 [ 3550.413906] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3551.704196] loop0: detected capacity change from 0 to 204800000 [ 3551.719443] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3552.992797] loop0: detected capacity change from 0 to 204800000 [ 3553.023889] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3553.035347] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3553.065350] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3553.069130] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3553.072620] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3553.076346] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3553.080060] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3553.085815] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3553.089672] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3553.093570] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3553.097128] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3553.100568] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3553.103914] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3553.108561] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3553.114275] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3553.118935] Lustre: 267640:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3555.730908] Lustre: server umount MGS complete [ 3556.023410] loop0: detected capacity change from 0 to 204800000 [ 3556.048897] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3556.234515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3556.250092] Lustre: lustre-MDT0000: Aborting client recovery [ 3556.558793] loop1: detected capacity change from 0 to 204800000 [ 3556.595751] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3556.635870] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3556.639454] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3556.643117] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3556.646738] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3556.650316] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3556.653570] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3556.656673] Lustre: 268069:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3556.663379] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3556.736687] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3556.761421] Lustre: lustre-MDT0001: Aborting client recovery [ 3557.067955] loop2: detected capacity change from 0 to 204800000 [ 3557.099972] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3557.134529] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3557.140935] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3557.146968] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3557.152925] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3557.158756] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3557.163423] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3557.167855] Lustre: 268150:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3557.232128] Lustre: Found index 0 for lustre-OST0000, updating log [ 3557.266382] Lustre: lustre-OST0000: Aborting recovery [ 3557.589854] loop3: detected capacity change from 0 to 204800000 [ 3557.631224] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3557.670673] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3557.674585] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3557.678534] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3557.682314] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3557.686242] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3557.689674] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3557.693318] Lustre: 268235:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3565.859216] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3565.859730] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3565.873003] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3565.874666] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3565.876498] LustreError: 268013:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 3566.882375] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3566.882768] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3568.434698] LustreError: 267980:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 201us: rc = -2 [ 3582.433187] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3582.433542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3582.440080] Lustre: Skipped 3 previous similar messages [ 3582.442618] Lustre: Skipped 2 previous similar messages [ 3582.447574] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3585.765108] Lustre: server umount lustre-MDT0000 complete [ 3586.069560] LustreError: 267962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775051981 with bad export cookie 4471570115265497118 [ 3586.071634] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3586.074742] LustreError: 267962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3587.551679] LustreError: 268176:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3587.551949] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3587.552229] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3587.552236] Lustre: Skipped 1 previous similar message [ 3587.559522] LustreError: 268176:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3587.563920] Lustre: Skipped 1 previous similar message [ 3591.583531] LustreError: 267979:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3591.583639] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3591.590579] LustreError: 267979:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3591.593154] Lustre: Skipped 2 previous similar messages [ 3592.323426] Lustre: server umount lustre-MDT0001 complete [ 3598.722910] Lustre: server umount lustre-OST0000 complete [ 3605.141558] Lustre: server umount lustre-OST0001 complete [ 3607.724110] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:00:03 (1775052003) [ 3616.618614] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 3619.630332] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3631.218624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3631.315040] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3631.326947] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3631.365971] Lustre: lustre-MDT0000: new disk, initializing [ 3631.390206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3631.392736] Lustre: Skipped 2 previous similar messages [ 3631.398398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3632.651943] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3636.671512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3636.716904] Lustre: 272274:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3636.743420] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3636.745284] Lustre: Skipped 1 previous similar message [ 3636.782736] Lustre: lustre-MDT0001: new disk, initializing [ 3636.818231] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3636.832133] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3636.835479] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3638.173787] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3641.223744] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3642.372949] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3644.524167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3644.622646] Lustre: lustre-OST0000: new disk, initializing [ 3644.624706] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3644.653700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3646.500321] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3649.536182] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3651.576468] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3651.579722] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3651.605777] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3657.200978] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3663.539860] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3671.877529] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3680.457644] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3728.357102] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3728.357107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3728.358874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3728.359983] LustreError: Skipped 1 previous similar message [ 3728.363607] Lustre: Skipped 1 previous similar message [ 3729.439126] Lustre: server umount lustre-OST0000 complete [ 3733.474235] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3733.475720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3733.478401] Lustre: Skipped 2 previous similar messages [ 3733.485818] Lustre: Skipped 1 previous similar message [ 3736.864222] Lustre: server umount lustre-MDT0000 complete [ 3738.091580] LustreError: 274758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775052133 with bad export cookie 4471570115265499827 [ 3738.095349] LustreError: 274758:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3738.096887] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3738.234700] Lustre: server umount lustre-MDT0001 complete [ 3741.063172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3742.314513] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3743.896385] Lustre: server umount MGS complete [ 3747.577192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3747.733714] LustreError: 277950:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3747.739985] LustreError: 277963:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3747.761749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3748.990335] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3751.801994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3753.208176] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3754.935983] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3756.064037] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3758.302926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3760.219682] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3761.952989] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3802.592084] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3802.595347] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3802.601556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3805.066805] Lustre: server umount lustre-OST0000 complete [ 3808.224148] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3808.227826] LustreError: Skipped 1 previous similar message [ 3812.372940] Lustre: server umount lustre-MDT0000 complete [ 3813.563396] LustreError: 277930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775052209 with bad export cookie 4471570115265501654 [ 3813.566558] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3813.568338] LustreError: 277930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3813.672888] Lustre: server umount lustre-MDT0001 complete [ 3816.963799] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 3817.973527] Key type lgssc unregistered [ 3818.100395] LNet: 281560:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3818.103153] LNetError: 281560:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3818.112303] LNet: Removed LNI 192.168.203.125@tcp [ 3818.413757] Key type .llcrypt unregistered [ 3818.414900] Key type ._llcrypt unregistered [ 3830.212589] Key type ._llcrypt registered [ 3830.215422] Key type .llcrypt registered [ 3830.256503] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 3836.541647] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 3836.877437] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3836.933632] alg: No test for adler32 (adler32-zlib) [ 3837.800986] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 3837.890730] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 3839.471162] Key type lgssc registered [ 3839.901561] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3851.559707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3851.566343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3852.652647] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3852.664800] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3852.700830] Lustre: lustre-MDT0000: new disk, initializing [ 3852.722902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3852.728641] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3853.896600] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3857.915910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3857.955636] Lustre: 285456:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3857.977880] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3857.979574] Lustre: Skipped 1 previous similar message [ 3858.013652] Lustre: lustre-MDT0001: new disk, initializing [ 3858.044287] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3858.055678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3858.059346] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3859.234674] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3862.166615] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3863.326847] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3865.495910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3865.596036] Lustre: lustre-OST0000: new disk, initializing [ 3865.598931] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3865.625860] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3867.319894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3867.324097] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3867.359309] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3867.401200] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3870.333453] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3873.606076] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3878.727198] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3883.266398] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3889.786713] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3934.177287] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3934.177887] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3934.179501] LustreError: Skipped 1 previous similar message [ 3934.185656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3934.187947] Lustre: Skipped 1 previous similar message [ 3936.592861] Lustre: server umount lustre-OST0000 complete [ 3939.296523] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3939.297084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3939.300125] Lustre: Skipped 2 previous similar messages [ 3939.303448] Lustre: Skipped 1 previous similar message [ 3944.416458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3944.418681] Lustre: Skipped 1 previous similar message [ 3949.536315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3949.538471] Lustre: Skipped 1 previous similar message [ 3952.095108] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3952.151737] Lustre: server umount lustre-MDT0000 complete [ 3953.330036] LustreError: 285447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775052349 with bad export cookie 16915250655887493200 [ 3953.332748] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3953.332863] LustreError: 285447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3953.439958] Lustre: server umount lustre-MDT0001 complete [ 3956.116078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3957.335994] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3958.848516] Lustre: server umount MGS complete [ 3962.554366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3962.726133] LustreError: 290940:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3962.747300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3964.040454] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3966.908558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3967.030823] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3968.367466] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3970.078393] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3971.260286] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3973.600902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3975.590761] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 3977.360131] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4013.024457] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4013.026582] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4013.030374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4014.050191] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4017.745287] Lustre: server umount lustre-OST0000 complete [ 4019.168366] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4019.172982] Lustre: Skipped 2 previous similar messages [ 4024.288636] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4024.290906] Lustre: Skipped 5 previous similar messages [ 4025.242671] Lustre: server umount lustre-MDT0000 complete [ 4026.490138] LustreError: 290908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775052422 with bad export cookie 16915250655887495027 [ 4026.493050] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4026.493786] LustreError: 290908:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4026.599316] Lustre: server umount lustre-MDT0001 complete [ 4029.779900] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 4030.734233] Key type lgssc unregistered [ 4030.861438] LNet: 294535:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4030.863715] LNetError: 294535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4030.873363] LNet: Removed LNI 192.168.203.125@tcp [ 4031.149257] Key type .llcrypt unregistered [ 4031.150477] Key type ._llcrypt unregistered [ 4037.624379] Key type ._llcrypt registered [ 4037.625696] Key type .llcrypt registered [ 4037.663685] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 4043.188924] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 4043.526481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4043.564475] alg: No test for adler32 (adler32-zlib) [ 4044.416433] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 4044.499470] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 4046.079156] Key type lgssc registered [ 4046.453015] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4054.101833] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 4058.964663] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4058.972258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4060.052216] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4060.064489] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4060.098818] Lustre: lustre-MDT0000: new disk, initializing [ 4060.121666] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4060.129662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4061.402213] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4065.424257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4065.526421] Lustre: lustre-OST0000: new disk, initializing [ 4065.529129] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4065.531793] Lustre: Skipped 1 previous similar message [ 4065.556291] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4067.397486] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4070.894706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4070.898689] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4070.908168] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4204.049487] hrtimer: interrupt took 3460260 ns [ 4427.233798] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4427.252249] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4432.356193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4432.819997] Lustre: server umount lustre-MDT0000 complete [ 4435.241172] LustreError: 297693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775052830 with bad export cookie 12315339658265871701 [ 4435.243812] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4435.252159] LustreError: 297693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4435.363727] Lustre: server umount lustre-OST0000 complete [ 4450.663494] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 4456.544679] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 4478.592104] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4478.732769] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4478.751403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4478.797933] Lustre: lustre-MDT0000: new disk, initializing [ 4478.844346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4478.862870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4481.360419] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4488.778413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4488.855779] Lustre: 302700:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4488.893803] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4488.898213] Lustre: Skipped 1 previous similar message [ 4488.961479] Lustre: lustre-MDT0001: new disk, initializing [ 4489.029181] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4489.070670] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4489.080259] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4491.432624] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4496.282331] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4498.483747] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4502.330842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4502.480422] Lustre: lustre-OST0000: new disk, initializing [ 4502.488405] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4502.538110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4505.794825] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4509.206196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4509.213366] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4509.278048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4510.405591] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4512.767975] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4512.896995] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4515.123484] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4515.255646] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4519.397267] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4519.399711] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4519.403198] LustreError: Skipped 1 previous similar message [ 4519.413988] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4519.419773] Lustre: Skipped 1 previous similar message [ 4522.563827] Lustre: server umount lustre-OST0000 complete [ 4525.024479] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4525.031787] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4525.049216] Lustre: Skipped 1 previous similar message [ 4525.052455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4529.632510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4529.633079] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4530.832545] Lustre: server umount lustre-MDT0000 complete [ 4532.906880] LustreError: 302691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775052928 with bad export cookie 12315339658268812492 [ 4532.910499] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4532.911857] LustreError: 302691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4533.077979] Lustre: server umount lustre-MDT0001 complete [ 4535.019062] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4536.057869] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 10:15:31 (1775052931) [ 4539.789839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4540.154106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4542.443946] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4545.517313] LustreError: 306604:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4545.538456] LustreError: 306604:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4547.714388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4550.340589] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4553.707635] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4556.077217] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4560.116447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4560.317660] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4560.323951] Lustre: Skipped 1 previous similar message [ 4563.333474] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4566.158695] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4570.016573] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4570.109832] Lustre: lustre-OST0001: new disk, initializing [ 4570.113392] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4573.290535] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4578.836313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4578.843280] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4578.868596] LustreError: 309842:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4578.927301] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4579.031235] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4579.941168] LustreError: 309850:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4579.945834] LustreError: 309850:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4580.963516] LustreError: 309850:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4580.970469] LustreError: 309850:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4581.080863] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9462346fc800.ost_server_uuid 50 [ 4581.962229] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9462346fc800.ost_server_uuid in FULL state after 0 sec [ 4583.011490] LustreError: 309850:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4583.019281] LustreError: 309850:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4583.908068] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9462346fc800.ost_server_uuid 50 [ 4584.690583] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9462346fc800.ost_server_uuid in FULL state after 0 sec [ 4586.099462] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4599.273121] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4599.276333] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4599.290085] Lustre: Skipped 1 previous similar message [ 4599.293874] Lustre: Skipped 1 previous similar message [ 4603.591372] Lustre: server umount lustre-OST0001 complete [ 4604.390135] LustreError: 308191:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4604.401033] LustreError: 308191:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4608.814247] Lustre: server umount lustre-OST0000 complete [ 4614.626073] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4614.628141] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4619.747558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4619.752481] Lustre: Skipped 5 previous similar messages [ 4620.834540] Lustre: server umount lustre-MDT0000 complete [ 4622.898911] LustreError: 308214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775053018 with bad export cookie 12315339658268813360 [ 4622.903727] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4622.909827] LustreError: 308214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4633.148611] Lustre: server umount lustre-MDT0001 complete [ 4639.433251] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 4641.155314] Key type lgssc unregistered [ 4641.342173] LNet: 311538:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4641.349332] LNetError: 311538:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4641.363764] LNet: Removed LNI 192.168.203.125@tcp [ 4641.847931] Key type .llcrypt unregistered [ 4641.849878] Key type ._llcrypt unregistered [ 4656.536879] Key type ._llcrypt registered [ 4656.538910] Key type .llcrypt registered [ 4656.610861] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 4657.122388] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4657.221075] alg: No test for adler32 (adler32-zlib) [ 4658.199456] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 4658.351881] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 4659.967340] Key type lgssc registered [ 4660.663576] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4665.483617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4665.499992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4666.795565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4668.971640] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4672.153957] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4676.224074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4676.474192] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4678.806476] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4683.906790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4684.087793] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4687.140314] LustreError: 314469:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4687.151616] LustreError: 314469:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4687.392620] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4690.460665] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4692.453136] LustreError: 314768:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4693.479653] LustreError: 314470:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4693.484603] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4693.490188] LustreError: 314470:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4694.007679] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4694.121163] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4696.973614] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4699.631710] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4700.017900] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4704.682426] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4704.805777] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4796.903361] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4796.905778] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4796.907075] LustreError: Skipped 1 previous similar message [ 4796.912681] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4796.916597] Lustre: Skipped 1 previous similar message [ 4799.363296] Lustre: server umount lustre-OST0001 complete [ 4802.020753] LustreError: 314470:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4802.028175] LustreError: 314470:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4805.088434] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4805.095862] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4805.105524] Lustre: Skipped 1 previous similar message [ 4805.111093] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4806.112279] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4806.116790] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4806.125569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4807.138298] LustreError: 314768:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4807.145670] LustreError: 314768:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4812.257383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4812.260350] Lustre: Skipped 2 previous similar messages [ 4816.352258] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4816.453652] Lustre: server umount lustre-OST0000 complete [ 4819.936176] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4819.939654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4819.946092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4819.948563] Lustre: Skipped 1 previous similar message [ 4824.713249] Lustre: server umount lustre-MDT0000 complete [ 4826.720844] LustreError: 312902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775053222 with bad export cookie 18291568848629259818 [ 4826.722669] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4826.727483] LustreError: 312902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4826.879909] Lustre: server umount lustre-MDT0001 complete [ 4831.694217] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 4833.145502] Key type lgssc unregistered [ 4833.303744] LNet: 317479:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4833.306902] LNetError: 317479:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4833.316968] LNet: Removed LNI 192.168.203.125@tcp [ 4833.739313] Key type .llcrypt unregistered [ 4833.740828] Key type ._llcrypt unregistered [ 4857.844715] Key type ._llcrypt registered [ 4857.846396] Key type .llcrypt registered [ 4857.904488] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 4858.340775] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4858.372138] alg: No test for adler32 (adler32-zlib) [ 4859.300441] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 4859.435221] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 4861.047183] Key type lgssc registered [ 4861.658830] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4866.434061] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4866.444230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4867.680639] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4869.675403] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4873.748609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4873.958830] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4875.988000] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4877.483447] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4880.478507] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4880.656026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4883.310777] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4883.748215] LustreError: 321180:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4883.754790] LustreError: 321180:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4887.138472] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4887.257280] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4889.922604] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4892.397357] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4892.402876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4898.989419] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4901.100343] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4915.167941] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4915.171580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4915.179989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4916.196598] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4916.199351] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4916.205001] Lustre: Skipped 2 previous similar messages [ 4916.208810] Lustre: Skipped 2 previous similar messages [ 4917.502335] Lustre: server umount lustre-MDT0000 complete [ 4919.118647] LustreError: 322674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775053314 with bad export cookie 11804579671111898259 [ 4919.121518] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4919.123182] LustreError: 322674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4919.269432] Lustre: server umount lustre-MDT0001 complete [ 4930.574408] Lustre: server umount lustre-OST0000 complete [ 4941.505995] Lustre: server umount lustre-OST0001 complete [ 4948.602175] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 4953.241534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4953.462206] LustreError: 324911:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4953.501485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4955.300396] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4958.688682] LustreError: 324912:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4959.219494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4961.055220] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4962.361330] Lustre: 326019:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4965.348188] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4965.516684] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4965.519873] Lustre: Skipped 1 previous similar message [ 4967.876808] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4969.637048] LustreError: 326373:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4971.411102] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4973.912685] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 4978.605355] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4978.607957] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4982.734378] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4984.213672] Lustre: 327861:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4994.019119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4994.026366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4994.028226] Lustre: Skipped 3 previous similar messages [ 4994.032861] Lustre: Skipped 4 previous similar messages [ 4996.790376] Lustre: server umount lustre-MDT0000 complete [ 4999.138319] LustreError: 324912:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4999.165321] LustreError: 324912:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5000.539222] LustreError: 324893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775053396 with bad export cookie 11804579671111899960 [ 5000.541098] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5000.549292] LustreError: 324893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5001.082152] Lustre: server umount lustre-MDT0001 complete [ 5015.268806] Lustre: server umount lustre-OST0000 complete [ 5020.130677] Lustre: 318990:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775053399/real 1775053400] req@ffff9d07d05a4380 x1861278242011392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775053415 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5020.205190] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5020.794145] Lustre: server umount lustre-OST0001 complete [ 5025.675478] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5027.452599] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5029.433936] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 10:23:44 (1775053424) [ 5053.037735] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 5058.668569] loop0: detected capacity change from 0 to 1099511627776 [ 5059.106623] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5059.110682] *** VALIDATE xfs *** [ 5059.121017] XFS (loop0): Mounting V5 Filesystem [ 5059.144380] XFS (loop0): Ending clean mount [ 5060.037933] loop1: detected capacity change from 0 to 18691697672192 [ 5115.360248] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 5122.199212] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 5145.926917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5146.058433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5146.078227] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5146.132263] Lustre: lustre-MDT0000: new disk, initializing [ 5146.175381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5146.178665] Lustre: Skipped 1 previous similar message [ 5146.189660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5148.628290] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5155.519174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5155.578566] Lustre: 332937:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5155.625096] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5155.628247] Lustre: Skipped 1 previous similar message [ 5155.670940] Lustre: lustre-MDT0001: new disk, initializing [ 5155.794873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5155.801836] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5158.697674] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5163.919360] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5166.144946] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5170.326241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5170.498306] Lustre: lustre-OST0000: new disk, initializing [ 5170.504482] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5171.849954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5171.857670] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5171.931703] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5174.065682] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5178.693612] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5181.167724] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5181.305586] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5183.608931] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5183.737677] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5187.554625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5187.556740] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5187.558232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5187.562824] Lustre: Skipped 1 previous similar message [ 5192.676329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5192.680253] Lustre: Skipped 2 previous similar messages [ 5199.327133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5199.419053] Lustre: server umount lustre-OST0000 complete [ 5201.888829] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5201.894640] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5201.901536] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5201.904051] Lustre: Skipped 2 previous similar messages [ 5216.223361] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5216.314956] Lustre: server umount lustre-MDT0000 complete [ 5218.276601] LustreError: 332949:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5218.290838] LustreError: 332949:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5218.767186] LustreError: 332929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775053614 with bad export cookie 11804579671111901514 [ 5218.769455] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5218.782080] LustreError: 332929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5219.024248] Lustre: server umount lustre-MDT0001 complete [ 5224.579862] XFS (loop0): Unmounting Filesystem [ 5227.873784] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 10:27:03 (1775053623) [ 5232.482213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5232.821512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5232.826826] Lustre: Skipped 2 previous similar messages [ 5235.623712] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5238.240487] LustreError: 337089:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5238.252052] LustreError: 337089:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5241.572026] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5244.710574] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5248.913293] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5252.048992] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5257.138524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5261.596752] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5265.398883] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5277.152513] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5277.155721] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5277.164281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5277.167911] Lustre: Skipped 2 previous similar messages [ 5277.188134] Lustre: Skipped 7 previous similar messages [ 5281.572374] Lustre: server umount lustre-OST0000 complete [ 5290.533979] Lustre: server umount lustre-MDT0000 complete [ 5292.516196] LustreError: 338200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5292.526882] LustreError: 338200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5293.514473] LustreError: 337807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775053689 with bad export cookie 11804579671111902396 [ 5293.516828] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5293.521406] LustreError: 337807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5293.780757] Lustre: server umount lustre-MDT0001 complete [ 5301.640799] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 5303.718538] Key type lgssc unregistered [ 5303.979244] LNet: 340361:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5303.988028] LNetError: 340361:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5304.000944] LNet: Removed LNI 192.168.203.125@tcp [ 5304.687503] Key type .llcrypt unregistered [ 5304.689121] Key type ._llcrypt unregistered [ 5323.980170] Key type ._llcrypt registered [ 5323.983621] Key type .llcrypt registered [ 5324.072868] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 5324.865553] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5324.878715] alg: No test for adler32 (adler32-zlib) [ 5325.866360] Lustre: Lustre: Build Version: 2.17.51_74_g79190eb [ 5326.046731] LNet: Added LNI 192.168.203.125@tcp [8/256/0/180] [ 5327.727183] Key type lgssc registered [ 5328.568039] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5333.923138] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5333.952867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5335.321547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5337.976880] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5343.528544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5343.785147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5346.645594] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5351.252357] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5353.871677] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5359.568609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5360.004627] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5365.293394] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5369.797932] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5376.481073] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5376.487325] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5376.498748] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5377.896763] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) [ 5378.306973] Lustre: server umount lustre-MDT0001 complete [ 5379.558194] LustreError: 341891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5379.561218] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5379.575066] LustreError: 341891:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5382.986535] LustreError: 343735:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5383.000508] LustreError: 343735:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5384.672439] LustreError: 342772:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5384.684902] LustreError: 342772:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5385.912779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5385.988828] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5385.993400] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5386.028693] Lustre: 341880:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5386.262058] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5389.119303] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5391.331606] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5393.900817] LustreError: 345150:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5393.906139] LustreError: 345150:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5393.913325] LustreError: 341883:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5393.952863] LustreError: 345153:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.203.125@tcp: cfg command failed: rc = -17 [ 5393.968091] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5393.968091] [ 5693.409868] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5693.410343] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5693.417219] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5693.427524] Lustre: Skipped 1 previous similar message [ 5698.528103] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5698.530670] Lustre: Skipped 1 previous similar message [ 5703.652709] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5703.655898] Lustre: Skipped 3 previous similar messages [ 5707.231145] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5707.315224] Lustre: server umount lustre-MDT0001 complete [ 5708.771178] LustreError: 341888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5708.778673] LustreError: 341888:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5712.957329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5713.016112] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5713.019562] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5713.047612] Lustre: 341879:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5713.224369] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5715.526987] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 5718.506241] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5723.123529] LustreError: 355960:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5723.130391] LustreError: 355960:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5723.140464] LustreError: 341883:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5723.144352] LustreError: 341883:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5723.191691] LustreError: 355964:0:(obd_config.c:760:class_setup()) Device 8 already setup (type osp) [ 5723.195672] LustreError: 355964:0:(obd_config.c:864:class_detach()) OBD device 8 still set up [ 5723.202059] LustreError: 355964:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.203.125@tcp: cfg command failed: rc = -17 [ 5723.207820] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.125@tcp [ 5723.207820] [ 6020.577883] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6020.578122] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6020.585264] Lustre: Skipped 1 previous similar message [ 6020.587765] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6020.589930] Lustre: Skipped 2 previous similar messages [ 6024.468940] Lustre: server umount lustre-MDT0001 complete [ 6025.697103] LustreError: 352192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6025.704775] LustreError: 352192:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6028.366253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6028.417250] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6028.419624] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6028.435633] Lustre: 341880:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6028.565670] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6030.064485] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 6033.894991] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6034.405186] LustreError: 367468:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6034.408078] LustreError: 367468:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6034.411054] LustreError: 341883:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6034.413339] LustreError: 341883:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6034.429674] LustreError: 367471:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6034.435053] LustreError: 367471:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.203.125@tcp: cfg command failed: rc = -114 [ 6034.438471] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6034.438471] [ 6346.208528] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6346.211185] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6346.211851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6346.211856] Lustre: Skipped 1 previous similar message [ 6346.222676] Lustre: Skipped 3 previous similar messages [ 6351.640348] Lustre: server umount lustre-MDT0000 complete [ 6353.029964] LustreError: 341871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775054748 with bad export cookie 3924338374501208115 [ 6353.032865] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6353.034817] LustreError: 341871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6356.448612] LustreError: 341888:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6356.448992] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6356.459037] LustreError: 341888:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6359.226702] Lustre: server umount lustre-MDT0001 complete [ 6360.867684] Lustre: server umount lustre-OST0000 complete [ 6364.454641] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 6367.934026] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 6382.512781] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 10:46:18 (1775054778) [ 6384.946127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6385.040632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6385.056117] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6385.100633] Lustre: lustre-MDT0000: new disk, initializing [ 6385.135847] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6385.143977] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6386.650687] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 6391.515261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6391.574608] Lustre: 381499:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6391.606965] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6391.610454] Lustre: Skipped 1 previous similar message [ 6391.654684] Lustre: lustre-MDT0001: new disk, initializing [ 6391.704186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6391.724754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6391.729914] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6393.411987] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 6397.094596] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6398.662284] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6401.472947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6401.590942] Lustre: lustre-OST0000: new disk, initializing [ 6401.594625] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6401.636262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6403.878771] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_default_debug -1 all [ 6407.170870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6407.175378] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6407.219985] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6407.314206] Lustre: DEBUG MARKER: oleg325-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6412.257355] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6412.257878] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6412.258772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6412.258777] Lustre: Skipped 7 previous similar messages [ 6412.261442] Lustre: Skipped 2 previous similar messages [ 6416.283950] Lustre: server umount lustre-MDT0000 complete [ 6417.378915] LustreError: 381505:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6417.385797] LustreError: 381505:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6417.675042] LustreError: 381491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775054813 with bad export cookie 3924338374501210054 [ 6417.676891] LustreError: 382288:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6417.678160] LustreError: MGC192.168.203.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6417.678878] LustreError: 381491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6417.795721] Lustre: server umount lustre-MDT0001 complete [ 6429.599886] Lustre: server umount lustre-OST0000 complete [ 6440.877016] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing set_hostid [ 6444.936584] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing load_modules_local [ 6456.204505] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6349 sec ========== 10:47:32 (1775054852) [ 6456.780567] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 10:47:32 (1775054852) === [ 6457.970379] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 10:47:34 (1775054854) === [ 6471.660993] Lustre: DEBUG MARKER: oleg325-server.virtnet: executing unload_modules_local [ 6472.887770] Key type lgssc unregistered [ 6473.031341] LNet: 389431:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6473.034818] LNetError: 389431:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6473.041695] LNet: Removed LNI 192.168.203.125@tcp [ 6473.422437] Key type .llcrypt unregistered [ 6473.424207] Key type ._llcrypt unregistered