[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 498599965 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f53f0-0x000f53ff] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5200 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D87 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C23 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BE3 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C97 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D27 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D5F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c23-0xbffe1c96] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c22] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c97-0xbffe1d26] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d27-0xbffe1d5e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d5f-0xbffe1d86] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002000] Switched APIC routing to physical x2apic. [ 0.002000] kvm-guest: setup PV IPIs [ 0.002000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.002000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.002025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.003014] pid_max: default: 32768 minimum: 301 [ 0.004159] LSM: Security Framework initializing [ 0.005056] Yama: becoming mindful. [ 0.006035] SELinux: Initializing. [ 0.007068] *** VALIDATE selinux *** [ 0.016314] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.021604] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.022145] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023000] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023096] *** VALIDATE tmpfs *** [ 0.025481] *** VALIDATE proc *** [ 0.027126] *** VALIDATE cgroup *** [ 0.028010] *** VALIDATE cgroup2 *** [ 0.029220] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.030110] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.031011] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.032034] Spectre V2 : User space: Vulnerable [ 0.033010] Speculative Store Bypass: Vulnerable [ 0.035766] debug: unmapping init [mem 0xffffffff85a59000-0xffffffff85a60fff] [ 0.038276] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.039858] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.040034] ... version: 2 [ 0.041017] ... bit width: 48 [ 0.042015] ... generic registers: 4 [ 0.043016] ... value mask: 0000ffffffffffff [ 0.044022] ... max period: 00007fffffffffff [ 0.045024] ... fixed-purpose events: 3 [ 0.046016] ... event mask: 000000070000000f [ 0.047420] rcu: Hierarchical SRCU implementation. [ 0.049855] smp: Bringing up secondary CPUs ... [ 0.050698] x86: Booting SMP configuration: [ 0.051048] .... node #0, CPUs: #1 #2 #3 [ 0.055099] smp: Brought up 1 node, 4 CPUs [ 0.057032] smpboot: Max logical packages: 1 [ 0.058017] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.241206] node 0 deferred pages initialised in 181ms [ 0.243172] devtmpfs: initialized [ 0.244263] x86/mm: Memory block size: 128MB [ 0.247746] gcov: version magic: 0x41383552 [ 0.251247] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.254101] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.256356] pinctrl core: initialized pinctrl subsystem [ 0.259207] [ 0.259783] ************************************************************* [ 0.262011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.263013] ** ** [ 0.266018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.268012] ** ** [ 0.270011] ** This means that this kernel is built to expose internal ** [ 0.272012] ** IOMMU data structures, which may compromise security on ** [ 0.275013] ** your system. ** [ 0.277012] ** ** [ 0.279014] ** If you see this message and you are not debugging the ** [ 0.282016] ** kernel, report this immediately to your vendor! ** [ 0.284012] ** ** [ 0.287028] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.289015] ************************************************************* [ 0.294309] NET: Registered protocol family 16 [ 0.297517] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.301094] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.303094] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.306164] cpuidle: using governor menu [ 0.307734] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.308823] PCI: Using configuration type 1 for base access [ 0.311185] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.320098] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.322051] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.326043] cryptd: max_cpu_qlen set to 1000 [ 0.329190] ACPI: Added _OSI(Module Device) [ 0.331013] ACPI: Added _OSI(Processor Device) [ 0.332012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.334014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.338636] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.346600] ACPI: Interpreter enabled [ 0.348065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.349013] ACPI: Using IOAPIC for interrupt routing [ 0.352120] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.355411] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.366048] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.368047] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.371023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.375102] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.380369] acpiphp: Slot [2] registered [ 0.382169] acpiphp: Slot [5] registered [ 0.384162] acpiphp: Slot [6] registered [ 0.385166] acpiphp: Slot [7] registered [ 0.387163] acpiphp: Slot [8] registered [ 0.389159] acpiphp: Slot [9] registered [ 0.390155] acpiphp: Slot [10] registered [ 0.392173] acpiphp: Slot [3] registered [ 0.394120] acpiphp: Slot [4] registered [ 0.396112] acpiphp: Slot [11] registered [ 0.397116] acpiphp: Slot [12] registered [ 0.399114] acpiphp: Slot [13] registered [ 0.401108] acpiphp: Slot [14] registered [ 0.402101] acpiphp: Slot [15] registered [ 0.403122] acpiphp: Slot [16] registered [ 0.405124] acpiphp: Slot [17] registered [ 0.407136] acpiphp: Slot [18] registered [ 0.408134] acpiphp: Slot [19] registered [ 0.410128] acpiphp: Slot [20] registered [ 0.411110] acpiphp: Slot [21] registered [ 0.413162] acpiphp: Slot [22] registered [ 0.415094] acpiphp: Slot [23] registered [ 0.416101] acpiphp: Slot [24] registered [ 0.418094] acpiphp: Slot [25] registered [ 0.419115] acpiphp: Slot [26] registered [ 0.423185] acpiphp: Slot [27] registered [ 0.424114] acpiphp: Slot [28] registered [ 0.426139] acpiphp: Slot [29] registered [ 0.427093] acpiphp: Slot [30] registered [ 0.429098] acpiphp: Slot [31] registered [ 0.430081] PCI host bridge to bus 0000:00 [ 0.432021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.435025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.437021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.440028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.443030] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.445027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.447193] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.450583] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.455292] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.465722] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.470429] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.473021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.475018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.478019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.482441] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.484747] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.488046] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.490785] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.496020] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.510019] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.515769] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.522217] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.528019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.536024] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.552019] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.562322] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.568035] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.575036] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.591037] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.604671] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.611018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.617020] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.633020] pci 0000:00:07.0: reg 0x20: [mem 0x38000000c000-0x38000000ffff 64bit pref] [ 0.651461] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.658018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.664019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.679023] pci 0000:00:08.0: reg 0x20: [mem 0x380000010000-0x380000013fff 64bit pref] [ 0.688977] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.697018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.703021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.717029] pci 0000:00:09.0: reg 0x20: [mem 0x380000014000-0x380000017fff 64bit pref] [ 0.728972] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.736016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.742022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.755019] pci 0000:00:0a.0: reg 0x20: [mem 0x380000018000-0x38000001bfff 64bit pref] [ 0.770533] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.774443] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.778418] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.780388] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.783215] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.788345] iommu: Default domain type: Passthrough [ 0.789387] SCSI subsystem initialized [ 0.791107] ACPI: bus type USB registered [ 0.792096] usbcore: registered new interface driver usbfs [ 0.794088] usbcore: registered new interface driver hub [ 0.796084] usbcore: registered new device driver usb [ 0.798207] pps_core: LinuxPPS API ver. 1 registered [ 0.800011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.804068] PTP clock support registered [ 0.805148] EDAC MC: Ver: 3.0.0 [ 0.807109] PCI: Using ACPI for IRQ routing [ 0.808974] NetLabel: Initializing [ 0.810013] NetLabel: domain hash size = 128 [ 0.814017] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.816082] NetLabel: unlabeled traffic allowed by default [ 0.817281] vgaarb: loaded [ 0.819306] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.820016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.824711] clocksource: Switched to clocksource kvm-clock [ 0.933880] VFS: Disk quotas dquot_6.6.0 [ 0.935540] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.938481] *** VALIDATE ramfs *** [ 0.939807] *** VALIDATE hugetlbfs *** [ 0.941481] pnp: PnP ACPI init [ 0.943867] pnp: PnP ACPI: found 6 devices [ 0.972079] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.975616] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.977911] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.980157] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.982564] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.985046] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 0.987948] NET: Registered protocol family 2 [ 0.990361] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.995109] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.998648] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.005038] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.009209] TCP: Hash tables configured (established 65536 bind 65536) [ 1.012668] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.016075] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.019208] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.022494] NET: Registered protocol family 1 [ 1.026370] RPC: Registered named UNIX socket transport module. [ 1.028700] RPC: Registered udp transport module. [ 1.030528] RPC: Registered tcp transport module. [ 1.032377] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.034975] NET: Registered protocol family 44 [ 1.036775] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.039294] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.041598] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.043882] PCI: CLS 0 bytes, default 64 [ 1.045514] Unpacking initramfs... [ 2.460887] debug: unmapping init [mem 0xffff9bb87cc54000-0xffff9bb87ffbffff] [ 2.464972] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.467385] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.470409] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.976793] Initialise system trusted keyrings [ 2.978580] Key type blacklist registered [ 2.981743] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.993026] zbud: loaded [ 2.996443] *** VALIDATE nfs *** [ 2.997891] *** VALIDATE nfs4 *** [ 2.999581] pstore: using deflate compression [ 3.004922] Platform Keyring initialized [ 3.111964] NET: Registered protocol family 38 [ 3.114236] Key type asymmetric registered [ 3.115666] Asymmetric key parser 'x509' registered [ 3.117644] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.120744] io scheduler mq-deadline registered [ 3.122487] io scheduler kyber registered [ 3.124341] io scheduler bfq registered [ 3.126423] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.129976] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.133032] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.136368] ACPI: Power Button [PWRF] [ 3.232886] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.323224] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.502163] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.593024] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.778044] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.807794] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.842024] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.853426] Non-volatile memory driver v1.3 [ 3.854893] Linux agpgart interface v0.103 [ 3.907617] virtio_blk virtio1: [vda] 133848 512-byte logical blocks (68.5 MB/65.4 MiB) [ 3.911968] vda: detected capacity change from 0 to 68530176 [ 3.932152] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.935021] vdb: detected capacity change from 0 to 1073741824 [ 3.954191] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.957281] vdc: detected capacity change from 0 to 2621440000 [ 3.973616] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.976473] vdd: detected capacity change from 0 to 2621440000 [ 3.992424] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.995260] vde: detected capacity change from 0 to 4294967296 [ 4.012421] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.015223] vdf: detected capacity change from 0 to 4294967296 [ 4.021563] libphy: Fixed MDIO Bus: probed [ 4.041935] usbcore: registered new interface driver usbserial_generic [ 4.044378] usbserial: USB Serial support registered for generic [ 4.046680] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.050771] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.052919] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.056361] mousedev: PS/2 mouse device common for all mice [ 4.060261] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.060848] rtc_cmos 00:05: RTC can wake from S4 [ 4.067197] rtc_cmos 00:05: registered as rtc0 [ 4.069664] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.069814] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.076079] intel_pstate: CPU model not supported [ 4.079209] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.082882] hid: raw HID events driver (C) Jiri Kosina [ 4.087861] usbcore: registered new interface driver usbhid [ 4.090027] usbhid: USB HID core driver [ 4.092795] drop_monitor: Initializing network drop monitor service [ 4.095334] Initializing XFRM netlink socket [ 4.097205] NET: Registered protocol family 10 [ 4.100855] Segment Routing with IPv6 [ 4.102306] NET: Registered protocol family 17 [ 4.107975] mpls_gso: MPLS GSO support [ 4.113301] RAS: Correctable Errors collector initialized. [ 4.115208] AVX version of gcm_enc/dec engaged. [ 4.117481] AES CTR mode by8 optimization enabled [ 4.216966] sched_clock: Marking stable (4216938253, 0)->(5175782111, -958843858) [ 4.221099] registered taskstats version 1 [ 4.223376] Loading compiled-in X.509 certificates [ 4.225593] zswap: loaded using pool lzo/zbud [ 4.251436] Key type big_key registered [ 4.264440] Key type encrypted registered [ 4.266409] ima: No TPM chip found, activating TPM-bypass! [ 4.268680] ima: Allocated hash algorithm: sha1 [ 4.270279] ima: No architecture policies found [ 4.272294] evm: Initialising EVM extended attributes: [ 4.274757] evm: security.selinux [ 4.276293] evm: security.ima [ 4.277801] evm: security.capability [ 4.278804] evm: HMAC attrs: 0x1 [ 4.281860] rtc_cmos 00:05: setting system clock to 2025-11-17 01:29:23 UTC (1763342963) [ 4.288723] debug: unmapping init [mem 0xffffffff86a03000-0xffffffff86bfffff] [ 4.292135] debug: unmapping init [mem 0xffffffff85782000-0xffffffff85a58fff] [ 4.301090] Write protecting the kernel read-only data: 28672k [ 4.304875] debug: unmapping init [mem 0xffffffff83e03000-0xffffffff83ffffff] [ 4.307849] debug: unmapping init [mem 0xffffffff84714000-0xffffffff847fffff] [ 4.365334] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.380079] systemd[1]: Detected virtualization kvm. [ 4.381994] systemd[1]: Detected architecture x86-64. [ 4.384030] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.414581] systemd[1]: No hostname configured. [ 4.415992] systemd[1]: Set hostname to . [ 4.417871] random: systemd: uninitialized urandom read (16 bytes read) [ 4.420081] systemd[1]: Initializing machine ID from random generator. [ 4.472477] random: ln: uninitialized urandom read (6 bytes read) [ 4.557669] random: systemd: uninitialized urandom read (16 bytes read) [ 4.560543] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.567065] systemd[1]: Reached target Local Encrypted Volumes. [ OK ] Reached target Local Encrypted Volumes. [ 4.574019] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Slices. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.277320] device-mapper: uevent: version 1.0.3 [ 5.279759] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.034206] random: fast init done [ 6.057430] virtio_net virtio0 ens2: renamed from eth0 [ 6.084420] scsi host0: ata_piix [ 6.096232] scsi host1: ata_piix [ 6.099315] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.103425] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.732198] dracut-initqueue[586]: RTNETLINK answers: File exists [ 10.784956] random: crng init done [ 10.786589] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.200640] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.359287] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.577992] SELinux: Disabled at runtime. [ 12.637274] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.643331] systemd[1]: Detected virtualization kvm. [ 12.644715] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.066250] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.069989] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.075120] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.078488] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.081328] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.088083] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.095621] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting POSIX Message Queue File System... [ 13.154237] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-getty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Reached target RPC Port Mapper. Mounting Kernel Debug File System... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.486129] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.821056] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.833396] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.920728] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.951730] EDAC sbridge: Ver: 1.1.2 [ 15.467800] Key type dns_resolver registered [ 15.764518] NFS: Registering the id_resolver key type [ 15.766531] Key type id_resolver registered [ 15.767683] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg158-server login: [ 47.922061] libcfs: loading out-of-tree module taints kernel. [ 47.963733] Key type ._llcrypt registered [ 47.965816] Key type .llcrypt registered [ 48.076360] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 63.407344] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 64.500922] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 64.508967] alg: No test for adler32 (adler32-zlib) [ 65.771811] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 66.275157] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 67.968057] Key type lgssc registered [ 69.210807] Lustre: Echo OBD driver; http://www.lustre.org/ [ 82.659679] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 83.755935] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 89.895135] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 95.139133] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 101.596378] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 115.446936] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 125.611364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 125.664271] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 125.684937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 126.863253] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 126.887564] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 126.952751] Lustre: lustre-MDT0000: new disk, initializing [ 127.041419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 127.057866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 129.635538] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 140.736487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 140.827836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 140.949782] Lustre: 6518:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 140.981180] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 140.984143] Lustre: Skipped 1 previous similar message [ 141.078460] Lustre: lustre-MDT0001: new disk, initializing [ 141.149622] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 141.171699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 141.181924] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 144.094891] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 148.322731] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 155.041088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 155.081965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 155.244825] Lustre: lustre-OST0000: new disk, initializing [ 155.249372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 155.297765] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 159.107601] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 159.763318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 159.769063] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 159.823334] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 169.006498] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 169.058641] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 169.123802] Lustre: lustre-OST0001: new disk, initializing [ 169.129313] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 169.189080] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 169.385588] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 169.393078] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 169.444471] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 173.499940] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 183.733636] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 186.715123] hrtimer: interrupt took 3114880 ns [ 188.943317] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 197.985503] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing check_logdir /tmp/testlogs/ [ 202.533392] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing yml_node [ 207.775907] Lustre: DEBUG MARKER: Client: 2.16.61.47 [ 210.625209] Lustre: DEBUG MARKER: MDS: 2.16.61.47 [ 213.288551] Lustre: DEBUG MARKER: OSS: 2.16.61.47 [ 214.958169] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Nov 16 20:32:52 EST 2025 [ 229.573228] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 230.746313] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 248.804625] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 248.808913] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 248.832779] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 249.318238] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 249.357397] Lustre: Skipped 1 previous similar message [ 249.359929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 249.362762] Lustre: Skipped 3 previous similar messages [ 252.096797] LustreError: 12222:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 252.315886] Lustre: server umount lustre-MDT0000 complete [ 255.212458] LustreError: 6511:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343214 with bad export cookie 12331871426365191070 [ 255.223658] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 259.554148] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 259.558800] LustreError: 6526:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 259.558926] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 259.562504] Lustre: Skipped 2 previous similar messages [ 259.597603] LustreError: 6526:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 260.575778] LustreError: 6526:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 260.592452] LustreError: 6526:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 261.501135] LustreError: 12424:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 261.517239] LustreError: 12424:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 261.662223] Lustre: server umount lustre-MDT0001 complete [ 264.862087] LustreError: 12626:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 264.870352] LustreError: 12626:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 264.936071] Lustre: server umount lustre-OST0000 complete [ 267.916587] LustreError: 12826:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 267.919971] LustreError: 12826:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 268.024583] Lustre: server umount lustre-OST0001 complete [ 273.043094] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 280.759174] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 288.255661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 292.568980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 297.075496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 302.349735] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 307.832489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 307.871329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 308.033425] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 308.057420] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 308.121242] Lustre: lustre-MDT0000: new disk, initializing [ 308.204926] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 308.219909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 311.158590] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 320.253350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 320.336579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 320.414497] Lustre: 15008:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 320.470689] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 320.474590] Lustre: Skipped 1 previous similar message [ 320.526599] Lustre: lustre-MDT0001: new disk, initializing [ 320.643395] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 320.684737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 320.702349] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 323.936827] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 329.950592] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 333.346562] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 338.812098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 338.903204] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 339.170843] Lustre: lustre-OST0000: new disk, initializing [ 339.178185] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 339.275570] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 341.040574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 341.055535] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 341.197639] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 344.965942] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 351.544119] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 355.049097] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 355.220158] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 358.537856] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 358.720599] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 361.961593] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 361.961653] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 361.967260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 361.967269] Lustre: Skipped 3 previous similar messages [ 361.982677] Lustre: Skipped 1 previous similar message [ 362.004168] LustreError: Skipped 1 previous similar message [ 366.531469] LustreError: 17950:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 366.545972] LustreError: 17950:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 366.686364] Lustre: server umount lustre-OST0000 complete [ 371.683149] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 371.691591] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 371.705146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 371.707963] Lustre: Skipped 1 previous similar message [ 376.031511] LustreError: 18152:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 376.041425] LustreError: 18152:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 376.141919] Lustre: server umount lustre-MDT0000 complete [ 377.314109] LustreError: 15881:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 379.649384] LustreError: 15798:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343338 with bad export cookie 12331871426365192603 [ 379.656869] LustreError: 15798:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 379.663488] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 380.010767] Lustre: server umount lustre-MDT0001 complete [ 384.526403] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:35:42 (1763343342) [ 389.747684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 390.196475] LustreError: 18879:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 390.210280] LustreError: 18879:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 390.271720] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 393.791509] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 395.749701] LustreError: 18879:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 400.153515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 403.358605] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 407.566143] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 410.464344] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 415.471630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 419.948783] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 424.151995] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 430.513642] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 436.195482] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 436.198893] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 436.203916] LustreError: 21180:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 436.206034] Lustre: Skipped 1 previous similar message [ 436.209786] LustreError: 21180:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 436.213163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 436.221775] Lustre: Skipped 2 previous similar messages [ 436.269226] Lustre: server umount lustre-MDT0000 complete [ 439.402414] LustreError: 18860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343398 with bad export cookie 12331871426365193485 [ 439.403719] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 439.426657] LustreError: 18860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 440.295041] LustreError: 18872:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 440.305238] LustreError: 18872:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 441.740916] Lustre: server umount lustre-MDT0001 complete [ 444.952033] Lustre: server umount lustre-OST0000 complete [ 459.957869] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 468.028816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 468.313972] LustreError: 22907:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 468.372536] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 468.380969] Lustre: Skipped 2 previous similar messages [ 471.309648] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 477.363116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 480.948212] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 483.126414] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 488.507546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 493.167770] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 500.073601] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 500.121348] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 500.261663] Lustre: lustre-OST0001: new disk, initializing [ 500.267728] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 501.814137] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 501.818419] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 501.945156] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 505.041186] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 512.955238] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 515.621901] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 533.987625] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 533.990812] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 533.993294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 533.993302] Lustre: Skipped 1 previous similar message [ 534.007090] Lustre: Skipped 4 previous similar messages [ 545.247141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 545.318695] LustreError: 26386:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 545.327341] LustreError: 26386:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 545.465032] Lustre: server umount lustre-MDT0000 complete [ 548.745303] LustreError: 22888:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343507 with bad export cookie 12331871426365194332 [ 548.746296] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 548.768832] LustreError: 22888:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 549.056796] Lustre: server umount lustre-MDT0001 complete [ 562.485979] Lustre: server umount lustre-OST0000 complete [ 565.476869] Lustre: 3661:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763343508/real 1763343508] req@ffff9bb8fc5d9f80 x1848999176009472/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763343524 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 565.943659] Lustre: server umount lustre-OST0001 complete [ 578.240468] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 585.544890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 585.790780] LustreError: 28164:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 585.804116] LustreError: 28164:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 588.413414] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 595.167557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 599.881662] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 603.045665] Lustre: 29282:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 608.902985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 609.306216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 609.309806] Lustre: Skipped 5 previous similar messages [ 615.118401] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 622.410199] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 627.433168] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 633.152934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 636.165401] Lustre: 31134:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 647.136944] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 647.148580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 647.161779] Lustre: Skipped 2 previous similar messages [ 647.172802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 647.179519] Lustre: Skipped 11 previous similar messages [ 651.443841] LustreError: 31338:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 651.446189] LustreError: 31338:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 651.551226] Lustre: server umount lustre-MDT0000 complete [ 653.796394] LustreError: 28163:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 653.819117] LustreError: 28163:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 654.265062] LustreError: 28901:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343613 with bad export cookie 12331871426365195627 [ 654.268390] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 654.279522] LustreError: 28901:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 654.550440] Lustre: server umount lustre-MDT0001 complete [ 668.214306] Lustre: server umount lustre-OST0000 complete [ 681.115602] Lustre: server umount lustre-OST0001 complete [ 687.732517] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:40:45 (1763343645) [ 711.843941] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 723.222350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 728.129684] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 736.505312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 740.196491] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 742.656812] Lustre: 35040:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 748.355197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 753.369266] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 759.214862] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 763.156582] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 775.158884] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 778.291840] Lustre: 36895:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 787.939614] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 787.948669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 787.973878] Lustre: Skipped 3 previous similar messages [ 787.990579] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 787.993374] Lustre: Skipped 4 previous similar messages [ 793.183687] LustreError: 37047:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 793.186932] LustreError: 37047:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 793.265388] Lustre: server umount lustre-MDT0000 complete [ 794.607775] LustreError: 33923:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 794.617943] LustreError: 33923:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 796.726742] LustreError: 33905:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343755 with bad export cookie 12331871426365196733 [ 796.736552] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 796.751655] LustreError: 33905:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 797.109914] Lustre: server umount lustre-MDT0001 complete [ 811.251453] Lustre: server umount lustre-OST0000 complete [ 824.548447] Lustre: server umount lustre-OST0001 complete [ 832.391857] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:43:10 (1763343790) [ 847.243906] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 857.277556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 861.619760] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 870.582411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 871.259255] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 871.280134] Lustre: Skipped 6 previous similar messages [ 874.854741] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 877.127521] Lustre: 40137:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 882.237621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 887.140882] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 895.885260] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 901.863918] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 909.704859] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 912.921327] Lustre: 41995:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 918.929213] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 918.932895] Lustre: Skipped 1 previous similar message [ 922.089442] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 922.091256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 922.097476] Lustre: Skipped 5 previous similar messages [ 922.105605] Lustre: Skipped 7 previous similar messages [ 927.278823] Lustre: server umount lustre-MDT0000 complete [ 930.467399] LustreError: 39000:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343889 with bad export cookie 12331871426365197839 [ 930.474430] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 930.477758] LustreError: 39000:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 936.777992] Lustre: server umount lustre-MDT0001 complete [ 944.579316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 954.851873] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xab239bb9a84992b5 [ 954.862986] Lustre: MGC192.168.201.158@tcp: Connection restored to 0@lo (at 0@lo) [ 958.872722] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 965.483328] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 966.729357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 970.654108] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 972.290178] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 972.292531] Lustre: Skipped 1 previous similar message [ 975.570301] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 978.386234] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 1 sec [ 982.246925] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 984.118874] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 992.740901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 995.373733] Lustre: server umount lustre-MDT0000 complete [ 998.858332] LustreError: 38999:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343958 with bad export cookie 12331871426365199029 [ 998.864357] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 998.872982] LustreError: 38999:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 999.255251] Lustre: server umount lustre-MDT0001 complete [ 1013.762827] Lustre: server umount lustre-OST0000 complete [ 1026.062770] Lustre: server umount lustre-OST0001 complete [ 1033.342693] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:46:31 (1763343991) [ 1046.131796] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1053.342993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1053.587314] LustreError: 46370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1053.598135] LustreError: 46370:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 32 previous similar messages [ 1056.232966] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1063.330788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1067.249094] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1069.762642] Lustre: 47487:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1076.412506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1083.051475] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1092.267979] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1097.635787] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1106.843128] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1117.889793] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1130.467294] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1132.198728] LustreError: 49563:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1132.202026] LustreError: 49563:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1132.311565] Lustre: server umount lustre-MDT0000 complete [ 1135.831669] LustreError: 46349:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344095 with bad export cookie 12331871426365199813 [ 1135.834723] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1135.844298] LustreError: 46349:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1136.130747] Lustre: server umount lustre-MDT0001 complete [ 1150.193137] Lustre: server umount lustre-OST0000 complete [ 1163.162549] Lustre: server umount lustre-OST0001 complete [ 1173.145114] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:48:50 (1763344130) [ 1175.800959] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1177.763600] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:48:55 (1763344135) [ 1186.712107] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1192.956326] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1200.244934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1205.209099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1211.092918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1211.164932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1211.370535] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1211.426434] Lustre: lustre-MDT0000: new disk, initializing [ 1211.535635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1214.702932] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1220.744859] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1225.727514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1225.773901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1225.965370] Lustre: lustre-OST0000: new disk, initializing [ 1225.968929] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1225.972345] Lustre: Skipped 1 previous similar message [ 1228.050554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1228.057245] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1228.095299] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1230.305650] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1235.782183] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1237.258690] Lustre: DEBUG MARKER: create test files [ 1262.777941] Lustre: server umount lustre-OST0000 complete [ 1276.617378] Lustre: server umount lustre-MDT0000 complete [ 1286.071602] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1288.650731] Key type lgssc unregistered [ 1288.960750] LNet: 54899:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1288.972983] LNetError: 54899:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1290.024221] LNet: Removed LNI 192.168.201.158@tcp [ 1290.912457] Key type .llcrypt unregistered [ 1290.917157] Key type ._llcrypt unregistered [ 1319.607205] Key type ._llcrypt registered [ 1319.608790] Key type .llcrypt registered [ 1319.806438] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1321.718707] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1321.727127] alg: No test for adler32 (adler32-zlib) [ 1322.774427] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 1323.096727] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1324.847186] Key type lgssc registered [ 1326.472469] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1337.117715] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1337.154068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1338.776465] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1343.620777] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1350.522207] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1359.280735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1359.599768] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1363.709091] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1363.914349] Lustre: 57510:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1363.945824] Lustre: 57510:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1363.960322] Lustre: 57510:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1363.981209] Lustre: 57510:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1363.996594] Lustre: 57510:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1364.014092] Lustre: 57510:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1366.149675] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1372.046576] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1377.272198] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1382.656397] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1389.544437] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1389.547520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1389.570457] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1391.527477] LustreError: 58253:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1391.652320] Lustre: server umount lustre-OST0000 complete [ 1395.460112] LustreError: 58455:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1395.474932] LustreError: 58455:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1395.719662] Lustre: server umount lustre-MDT0000 complete [ 1405.620971] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1408.407413] Key type lgssc unregistered [ 1408.714725] LNet: 58987:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1408.722355] LNetError: 58987:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1409.770025] LNet: Removed LNI 192.168.201.158@tcp [ 1410.636448] Key type .llcrypt unregistered [ 1410.637979] Key type ._llcrypt unregistered [ 1438.792375] Key type ._llcrypt registered [ 1438.796894] Key type .llcrypt registered [ 1438.930969] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1440.132412] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1440.150988] alg: No test for adler32 (adler32-zlib) [ 1441.186492] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 1441.385383] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1443.031177] Key type lgssc registered [ 1444.212559] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1451.458367] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1451.490623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1452.946497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1456.851399] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1462.211509] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1468.839848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1469.300161] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1475.382303] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1478.514695] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1478.637294] Lustre: 61672:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1478.653535] Lustre: 61672:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1478.668716] Lustre: 61672:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1478.685383] Lustre: 61672:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1478.716634] Lustre: 61672:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1478.723757] Lustre: 61672:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1481.403969] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1486.759768] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1499.106575] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1499.109118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1499.223103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1500.816610] LustreError: 62366:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1500.942447] Lustre: server umount lustre-OST0000 complete [ 1505.975869] LustreError: 62568:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1505.981821] LustreError: 62568:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1506.267933] Lustre: server umount lustre-MDT0000 complete [ 1517.424474] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 1520.432623] Key type lgssc unregistered [ 1520.666235] LNet: 63100:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1520.673755] LNetError: 63100:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1520.689596] LNet: Removed LNI 192.168.201.158@tcp [ 1521.543718] Key type .llcrypt unregistered [ 1521.558208] Key type ._llcrypt unregistered [ 1538.463973] Key type ._llcrypt registered [ 1538.465640] Key type .llcrypt registered [ 1538.565869] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1552.603354] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1553.240346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1553.259196] alg: No test for adler32 (adler32-zlib) [ 1554.248372] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 1554.419904] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 1556.097762] Key type lgssc registered [ 1557.159844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1566.177068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1572.220310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1578.287202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1584.455830] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1591.622912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1591.707963] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1591.737856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1593.073029] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1593.106183] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1593.219949] Lustre: lustre-MDT0000: new disk, initializing [ 1593.327523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1593.358405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1597.274167] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1607.776495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1607.866445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1607.975756] Lustre: 66226:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1608.091465] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1608.094361] Lustre: Skipped 1 previous similar message [ 1608.137183] Lustre: lustre-MDT0001: new disk, initializing [ 1608.273842] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1608.362106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1608.380232] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1612.322815] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1619.017887] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1622.513836] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1629.373235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1629.480257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1629.837571] Lustre: lustre-OST0000: new disk, initializing [ 1629.844090] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1629.957182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1635.615713] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1637.994346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1638.000214] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1638.142449] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1642.952895] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1647.019331] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1647.207901] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1650.547235] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1650.715798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1653.729214] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1653.732514] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1653.735467] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1653.744964] Lustre: Skipped 1 previous similar message [ 1653.758625] Lustre: Skipped 1 previous similar message [ 1658.231071] LustreError: 69169:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1658.314779] Lustre: server umount lustre-OST0000 complete [ 1663.974994] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1663.989143] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1663.993198] Lustre: Skipped 1 previous similar message [ 1667.311919] LustreError: 69370:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1667.315327] LustreError: 69370:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1667.369189] Lustre: server umount lustre-MDT0000 complete [ 1670.271037] LustreError: 66219:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344629 with bad export cookie 3027987682666410880 [ 1670.278589] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1670.282745] LustreError: 66219:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1670.431230] LustreError: 69574:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1670.434945] LustreError: 69574:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1670.562719] Lustre: server umount lustre-MDT0001 complete [ 1678.024173] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:57:15 (1763344635) [ 1686.464931] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1697.518522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1723.993807] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 1735.284750] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 1745.844807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1753.031745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1759.814087] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1766.134261] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1772.936148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1772.987834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1773.244905] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1773.272386] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1773.332799] Lustre: lustre-MDT0000: new disk, initializing [ 1773.459775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1773.478453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1777.057645] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1786.769582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1786.837934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1786.908282] Lustre: 73827:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1786.978712] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1786.982158] Lustre: Skipped 1 previous similar message [ 1787.061234] Lustre: lustre-MDT0001: new disk, initializing [ 1787.147526] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1787.175070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1787.180524] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1790.626167] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1797.021055] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1800.737490] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1806.401973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1806.450928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1806.641312] Lustre: lustre-OST0000: new disk, initializing [ 1806.647347] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1806.731962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1811.742803] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1813.055884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1813.070408] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1813.160572] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1818.508427] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1821.884375] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1822.087508] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1825.623954] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1825.787557] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1828.853110] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1828.859961] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1828.865245] LustreError: Skipped 1 previous similar message [ 1828.868363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1828.868371] Lustre: Skipped 1 previous similar message [ 1833.529143] LustreError: 76768:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1833.538819] LustreError: 76768:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1833.614766] Lustre: server umount lustre-OST0000 complete [ 1838.564095] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1838.567867] LustreError: Skipped 1 previous similar message [ 1838.570867] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1838.593035] Lustre: Skipped 1 previous similar message [ 1838.599953] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1838.603626] Lustre: Skipped 1 previous similar message [ 1842.786130] LustreError: 76969:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1842.792336] LustreError: 76969:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1842.866611] Lustre: server umount lustre-MDT0000 complete [ 1844.193665] LustreError: 73834:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1844.218889] LustreError: 73834:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1846.123989] LustreError: 73818:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344805 with bad export cookie 3027987682666411762 [ 1846.132126] LustreError: 73818:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1846.132873] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1846.392523] Lustre: server umount lustre-MDT0001 complete [ 1853.591173] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 21:00:11 (1763344811) [ 1859.312504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1859.624469] LustreError: 77841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1859.694487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1863.321063] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1865.190274] LustreError: 77841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1869.285321] LustreError: 77841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1872.189198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1875.612223] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1880.948645] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1884.040960] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1889.784348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1894.961305] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1899.291215] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1901.937342] Lustre: 80178:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting b44c6ebd-3843-4149-8f3c-75f84730f792 at adminstrative request [ 1902.566622] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1902.588091] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1912.745584] Lustre: MGS: Client b44c6ebd-3843-4149-8f3c-75f84730f792 (at 0@lo) reconnecting [ 1912.759413] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x2a0592c12196fa64 to 0x2a0592c12196fcfd [ 1912.806651] Lustre: MGC192.168.201.158@tcp: Connection restored to 0@lo (at 0@lo) [ 1937.182407] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1937.244848] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1937.348686] Lustre: lustre-OST0001: new disk, initializing [ 1937.351367] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1937.420802] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1937.430735] Lustre: Skipped 2 previous similar messages [ 1942.278537] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 1947.212866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1947.224818] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1947.321390] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1948.226332] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1952.740219] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1952.743094] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1952.748653] LustreError: Skipped 1 previous similar message [ 1952.750264] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1952.755058] Lustre: Skipped 2 previous similar messages [ 1952.764487] Lustre: Skipped 3 previous similar messages [ 1956.469716] LustreError: 81334:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1956.474432] LustreError: 81334:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1956.520318] Lustre: server umount lustre-OST0001 complete [ 1957.862900] LustreError: 81150:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1957.882922] LustreError: 81150:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1960.930459] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1960.939585] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1960.967125] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1968.102556] LustreError: 80204:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1968.114320] LustreError: 80204:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1974.239171] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1974.284181] LustreError: 81537:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1974.297172] LustreError: 81537:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1974.394527] Lustre: server umount lustre-OST0000 complete [ 1978.338650] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1978.348644] Lustre: Skipped 1 previous similar message [ 1978.351464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1978.357889] Lustre: Skipped 7 previous similar messages [ 1983.645839] Lustre: server umount lustre-MDT0000 complete [ 1986.979303] LustreError: 77821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344946 with bad export cookie 3027987682666413309 [ 1986.991429] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1987.000709] LustreError: 77821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1987.391712] Lustre: server umount lustre-MDT0001 complete [ 1996.735466] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 21:02:34 (1763344954) [ 1998.490573] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2000.142958] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 21:02:37 (1763344957) [ 2001.736234] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2003.392811] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 21:02:41 (1763344961) [ 2005.236697] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2006.952552] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 21:02:44 (1763344964) [ 2008.359133] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2012.567147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2013.403595] loop: module loaded [ 2013.409805] loop0: detected capacity change from 0 to 4294967296 [ 2013.445027] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2014.887180] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2017.527825] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2020.930330] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2027.390747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2030.432922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2033.389307] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2036.140664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2041.525560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2041.573832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2041.599543] LustreError: 84515:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2041.607351] LustreError: 84515:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2041.616948] LustreError: 84515:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2041.622331] LustreError: 84515:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2055.219835] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2062.610081] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2071.071653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2076.246818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2082.663074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2088.957872] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2095.551706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2095.616446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2095.901838] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2095.941543] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2095.997449] Lustre: lustre-MDT0000: new disk, initializing [ 2096.072828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2096.090783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2100.242754] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2109.473077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2109.544459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2109.633418] Lustre: 87357:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2109.863168] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2113.211706] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2119.508670] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2122.681269] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2127.875350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2127.940765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2132.090437] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2134.562845] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2134.627712] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2137.843555] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2140.874537] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2141.006083] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2143.716355] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2143.934624] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2149.858406] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2149.859501] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2149.878486] Lustre: Skipped 1 previous similar message [ 2149.893970] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2149.898170] Lustre: Skipped 3 previous similar messages [ 2151.442484] LustreError: 90296:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2151.452322] LustreError: 90296:0:(obd_class.h:479:obd_check_dev()) Skipped 21 previous similar messages [ 2151.579763] Lustre: server umount lustre-OST0000 complete [ 2160.909996] Lustre: server umount lustre-MDT0000 complete [ 2163.624709] LustreError: 87348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763345122 with bad export cookie 3027987682666413911 [ 2163.627280] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2163.639337] LustreError: 87348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2163.940912] Lustre: server umount lustre-MDT0001 complete [ 2170.393763] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 21:05:28 (1763345128) [ 2175.029152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2175.324930] LustreError: 91370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2175.334954] LustreError: 91370:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2178.497528] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2184.854766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2188.361395] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2192.765966] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2195.579862] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2201.189373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2206.010727] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2210.400431] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2213.862137] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2214.039701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2217.264897] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2217.426502] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2223.334945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2223.418606] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2223.526658] Lustre: lustre-OST0001: new disk, initializing [ 2223.528795] Lustre: Skipped 2 previous similar messages [ 2223.534826] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2223.537713] Lustre: Skipped 3 previous similar messages [ 2228.831825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2228.842328] Lustre: Skipped 2 previous similar messages [ 2228.848489] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2228.934559] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2229.517551] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2236.053244] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2240.723895] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2240.933627] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2244.499260] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2244.666327] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2261.183714] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2287.114824] Lustre: Failing over lustre-MDT0000 [ 2287.441189] LustreError: 96277:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2287.444867] LustreError: 96277:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2287.582732] Lustre: server umount lustre-MDT0000 complete [ 2287.583826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2287.587869] LustreError: Skipped 1 previous similar message [ 2287.589716] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2287.605429] Lustre: Skipped 3 previous similar messages [ 2287.611030] LustreError: 92082:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2287.617134] LustreError: 92082:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2306.017382] Lustre: 64438:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763345249/real 1763345249] req@0000000054b7648f x1849000736310016/t0(0) o400->MGC192.168.201.158@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1763345265 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2306.054494] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2309.845976] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2309.847927] LDISKFS-fs (dm-0): recovery complete [ 2309.857268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2316.275077] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x2a0592c12197c76a [ 2316.281957] Lustre: MGC192.168.201.158@tcp: Connection restored to 0@lo (at 0@lo) [ 2316.492526] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2316.498160] Lustre: Skipped 6 previous similar messages [ 2316.534182] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2318.502949] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2318.506502] Lustre: 91364:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2319.586383] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2321.896887] Lustre: 91369:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2321.913778] Lustre: 91369:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 2321.922647] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2321.987338] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2322.295121] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2322.617754] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2322.628695] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2322.959140] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2322.963761] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2322.968912] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2322.985606] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2323.640750] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2323.651943] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2323.976932] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2323.981689] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2325.268051] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2325.274533] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2325.910333] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2325.931806] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2326.255909] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2326.275031] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2329.445839] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2329.457622] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2330.097660] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2330.104260] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2330.423088] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2330.426788] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2337.705196] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2337.712246] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 2338.344711] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2338.355182] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 2338.679140] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2338.682334] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 2354.019304] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2354.046987] Lustre: 97028:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 50 previous similar messages [ 2354.376049] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2354.387067] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 49 previous similar messages [ 2354.711167] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2354.741105] LustreError: 97028:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 49 previous similar messages [ 2378.311203] Lustre: 97028:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2378.316403] Lustre: 97028:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2378.328518] Lustre: 97028:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0c830895-c5f7-4d93-97b3-2b950392fb87@192.168.201.58@tcp [ 2378.350398] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2378.363856] Lustre: 97028:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000ad2baf74 x1849000721452032/t0(8589934773) o101->0c830895-c5f7-4d93-97b3-2b950392fb87@192.168.201.58@tcp:568/0 lens 592/0 e 0 to 0 dl 1763345348 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2378.395459] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2378.415425] Lustre: lustre-MDT0000: Denying connection for new client 0c830895-c5f7-4d93-97b3-2b950392fb87 (at 192.168.201.58@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2378.431654] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2378.502132] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2378.517439] Lustre: Skipped 2 previous similar messages [ 2378.521146] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2378.559062] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2378.562465] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2389.476633] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2389.481855] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2389.485832] Lustre: Skipped 4 previous similar messages [ 2394.819921] Lustre: server umount lustre-OST0000 complete [ 2398.690423] LustreError: 92967:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2398.697684] LustreError: 92967:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 36 previous similar messages [ 2404.246732] Lustre: server umount lustre-OST0001 complete [ 2421.215148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2421.342915] LustreError: 98095:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2421.353589] LustreError: 98095:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2421.437078] Lustre: server umount lustre-MDT0000 complete [ 2425.313330] LustreError: 98096:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763345384 with bad export cookie 3027987682666465130 [ 2425.320195] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2425.329209] LustreError: 98096:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2425.827418] Lustre: server umount lustre-MDT0001 complete [ 2434.599753] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 21:09:52 (1763345392) [ 2442.586622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2442.591200] Lustre: *** cfs_fail_loc=197, val=0*** [ 2458.207907] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2463.327465] LustreError: 98990:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.158@tcp: failed processing log, type 4: rc = -110 [ 2468.464862] Lustre: *** cfs_fail_loc=197, val=0*** [ 2468.466799] Lustre: Skipped 65 previous similar messages [ 2493.518955] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2498.090421] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2499.798740] Lustre: server umount lustre-OST0000 complete [ 2507.004910] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 21:11:04 (1763345464) [ 2527.153600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2527.594369] LustreError: 100499:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2527.603235] LustreError: 100499:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2531.234650] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2538.089739] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2542.416669] LustreError: 100529:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2542.447845] LustreError: 100529:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2542.461806] LustreError: 100529:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 15, retries 0, failed: rc = -5 [ 2542.822276] Lustre: server umount lustre-MDT0000 complete [ 2557.928214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2571.144767] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2578.495557] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2585.799565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2591.204196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2596.144042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2601.605719] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2607.363663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2607.428176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2607.611168] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2607.645439] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2607.711648] Lustre: lustre-MDT0000: new disk, initializing [ 2607.811624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2607.814302] Lustre: Skipped 2 previous similar messages [ 2607.840945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2610.946717] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2620.237852] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2620.302546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2620.360125] Lustre: 104569:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2620.591086] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2624.202099] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2632.260598] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2635.976973] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2643.412790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2643.495098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2649.979093] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2652.859988] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2657.587436] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2662.321693] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2662.518411] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2665.883749] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2666.118034] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2668.513168] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2668.527211] LustreError: Skipped 2 previous similar messages [ 2668.533416] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2668.539534] Lustre: Skipped 9 previous similar messages [ 2668.543666] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2668.546540] Lustre: Skipped 13 previous similar messages [ 2682.336306] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2682.379096] LustreError: 107512:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2682.389113] LustreError: 107512:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 2682.552722] Lustre: server umount lustre-OST0000 complete [ 2691.823250] Lustre: server umount lustre-MDT0000 complete [ 2694.965308] LustreError: 104561:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763345654 with bad export cookie 3027987682666475721 [ 2694.970778] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2694.972797] LustreError: 104561:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2695.174832] Lustre: server umount lustre-MDT0001 complete [ 2701.799036] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 21:14:19 (1763345659) [ 2708.888466] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 2711.241140] Key type lgssc unregistered [ 2711.514813] LNet: 108644:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2711.524650] LNetError: 108644:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2711.541705] LNet: Removed LNI 192.168.201.158@tcp [ 2712.326903] Key type .llcrypt unregistered [ 2712.328424] Key type ._llcrypt unregistered [ 2731.783145] Key type ._llcrypt registered [ 2731.784798] Key type .llcrypt registered [ 2731.911243] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2746.978439] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2747.834579] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2747.856125] alg: No test for adler32 (adler32-zlib) [ 2748.792976] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 2748.995313] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 2750.655366] Key type lgssc registered [ 2751.792178] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2760.703853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2767.197072] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2772.205000] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2778.182356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2783.156910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2789.372252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2789.444063] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2789.464404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2790.691659] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2790.723420] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2790.801443] Lustre: lustre-MDT0000: new disk, initializing [ 2790.896727] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2790.915324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2794.111887] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2803.618561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2803.711512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2803.935772] Lustre: lustre-OST0000: new disk, initializing [ 2803.949639] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2803.958401] Lustre: Skipped 1 previous similar message [ 2804.051097] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2808.807248] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2809.521638] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2809.538628] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2809.590187] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2818.304719] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2818.380982] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2818.502661] Lustre: lustre-OST0001: new disk, initializing [ 2818.521568] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2818.619474] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2823.809746] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2828.352336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2828.359952] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2828.389619] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2833.378936] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2833.390739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2833.394315] Lustre: Skipped 1 previous similar message [ 2838.171337] LustreError: 114220:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2838.318341] Lustre: server umount lustre-MDT0000 complete [ 2843.633186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2847.420065] LustreError: 112779:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763345806 with bad export cookie 11525001590175799361 [ 2847.428952] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2847.437460] LustreError: 112779:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2847.509142] LustreError: 114815:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2847.512220] LustreError: 114815:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2847.542906] Lustre: server umount lustre-OST0000 complete [ 2850.808549] LustreError: 115015:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2850.812808] LustreError: 115015:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2850.987270] Lustre: server umount lustre-OST0001 complete [ 2859.376365] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 21:16:57 (1763345817) [ 2867.349656] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2876.006903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2877.028123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2882.137773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2883.171650] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2888.919282] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 21:17:26 (1763345846) [ 2898.707304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2900.215892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2901.972178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2903.599782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2904.433571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2906.132728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2919.672243] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 2926.409936] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 2934.031088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2939.892531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2945.190658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2950.466180] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2956.962494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2957.033492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2957.319866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2957.364763] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2957.477929] Lustre: lustre-MDT0000: new disk, initializing [ 2957.575936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2957.588492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2960.944660] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2970.807468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2970.870152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2970.994757] Lustre: 120344:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2971.052853] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2971.056128] Lustre: Skipped 1 previous similar message [ 2971.149632] Lustre: lustre-MDT0001: new disk, initializing [ 2971.259639] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2971.307742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2971.320575] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2975.861118] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 2983.951761] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2988.117521] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2995.423322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2995.534343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2995.847902] Lustre: lustre-OST0000: new disk, initializing [ 2995.860424] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2995.962090] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3001.015361] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3003.422715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3003.426941] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3003.510922] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3007.400893] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3011.187980] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3011.380250] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3015.234798] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3015.523710] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3018.727359] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3018.731728] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3018.745853] Lustre: Skipped 1 previous similar message [ 3018.756531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3018.769715] Lustre: Skipped 1 previous similar message [ 3023.436246] LustreError: 123283:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3023.454969] LustreError: 123283:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3023.590228] Lustre: server umount lustre-OST0000 complete [ 3028.960360] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3028.969819] Lustre: Skipped 1 previous similar message [ 3028.975436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3028.980606] Lustre: Skipped 1 previous similar message [ 3034.086809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3034.089319] Lustre: Skipped 1 previous similar message [ 3039.201217] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3039.215939] Lustre: Skipped 1 previous similar message [ 3042.271173] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3042.348394] LustreError: 123484:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3042.351053] LustreError: 123484:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3042.443452] Lustre: server umount lustre-MDT0000 complete [ 3046.384663] LustreError: 120335:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346005 with bad export cookie 11525001590175800579 [ 3046.398022] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3046.400838] LustreError: 120335:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3046.702260] Lustre: server umount lustre-MDT0001 complete [ 3055.511436] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 21:20:13 (1763346013) [ 3062.368684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3062.811931] LustreError: 124359:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3062.855193] LustreError: 124359:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3062.942201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3066.791965] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3068.385614] LustreError: 124359:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3073.513451] LustreError: 124359:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3075.835463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3081.779990] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3088.215984] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3092.315619] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3099.796201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3100.057509] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3100.066252] Lustre: Skipped 1 previous similar message [ 3105.187166] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3110.625803] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3115.070392] Lustre: *** cfs_fail_loc=159, val=0*** [ 3131.616177] Lustre: lustre-MDT0000: Client 6edf9e2d-0750-4e70-9bab-2be0f4713cc1 (at 192.168.201.58@tcp) reconnecting [ 3133.155202] Lustre: *** cfs_fail_loc=159, val=0*** [ 3133.166865] Lustre: Skipped 5 previous similar messages [ 3149.531085] Lustre: lustre-MDT0000: Client 6edf9e2d-0750-4e70-9bab-2be0f4713cc1 (at 192.168.201.58@tcp) reconnecting [ 3167.721734] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3167.732297] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3167.751905] Lustre: Skipped 1 previous similar message [ 3167.762374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3169.046297] LustreError: 126850:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3169.049162] LustreError: 126850:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3169.224657] Lustre: server umount lustre-OST0000 complete [ 3174.891425] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3184.100397] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3184.120656] Lustre: Skipped 5 previous similar messages [ 3184.228195] Lustre: server umount lustre-MDT0000 complete [ 3188.594213] LustreError: 124340:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346147 with bad export cookie 11525001590175801461 [ 3188.611383] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3188.619194] LustreError: 124340:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3188.762133] LustreError: 127255:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3188.772691] LustreError: 127255:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3188.954820] Lustre: server umount lustre-MDT0001 complete [ 3199.398619] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3202.798632] Key type lgssc unregistered [ 3203.113747] LNet: 127788:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3203.129784] LNetError: 127788:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3204.201765] LNet: Removed LNI 192.168.201.158@tcp [ 3205.094202] Key type .llcrypt unregistered [ 3205.098843] Key type ._llcrypt unregistered [ 3230.932337] Key type ._llcrypt registered [ 3230.939501] Key type .llcrypt registered [ 3231.048511] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3231.924747] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3232.001886] alg: No test for adler32 (adler32-zlib) [ 3233.025479] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 3233.272413] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3234.983182] Key type lgssc registered [ 3236.256336] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3244.888598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3244.914910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3246.485334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3250.995532] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3251.687122] LustreError: 129196:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3251.723286] LustreError: 129196:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3255.794801] LustreError: 129196:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3260.412907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3260.881913] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3265.464501] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3271.711508] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3275.922182] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3282.826524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3283.261553] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3289.035051] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3291.439428] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3294.472548] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3303.018198] Lustre: *** cfs_fail_loc=159, val=0*** [ 3319.533339] Lustre: lustre-MDT0000: Client 0e91fce9-ed70-4414-82c0-49a571b74e38 (at 192.168.201.58@tcp) reconnecting [ 3321.971676] Lustre: *** cfs_fail_loc=159, val=0*** [ 3338.480134] Lustre: lustre-MDT0001: Client 0e91fce9-ed70-4414-82c0-49a571b74e38 (at 192.168.201.58@tcp) reconnecting [ 3340.083396] Lustre: *** cfs_fail_loc=159, val=0*** [ 3340.085028] Lustre: Skipped 3 previous similar messages [ 3356.423086] Lustre: lustre-MDT0001: Client 0e91fce9-ed70-4414-82c0-49a571b74e38 (at 192.168.201.58@tcp) reconnecting [ 3362.419135] Lustre: 131902:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3365.755067] Lustre: *** cfs_fail_loc=159, val=0*** [ 3365.760025] Lustre: Skipped 4 previous similar messages [ 3380.941857] Lustre: lustre-MDT0000: Client c8219b8f-efa7-4d56-9a22-f1deb02a884c (at 192.168.201.58@tcp) reconnecting [ 3383.108704] Lustre: *** cfs_fail_loc=159, val=0*** [ 3383.110462] Lustre: Skipped 14 previous similar messages [ 3399.402810] Lustre: lustre-MDT0000: Client c8219b8f-efa7-4d56-9a22-f1deb02a884c (at 192.168.201.58@tcp) reconnecting [ 3400.796277] Lustre: 132200:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3419.618810] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3419.620528] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3419.627777] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3419.667444] Lustre: Skipped 1 previous similar message [ 3424.738806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3424.750072] Lustre: Skipped 2 previous similar messages [ 3425.458162] LustreError: 132324:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3425.584779] Lustre: server umount lustre-OST0000 complete [ 3429.856078] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3429.857575] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3429.871223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3429.874432] Lustre: Skipped 1 previous similar message [ 3434.976941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3434.992596] Lustre: Skipped 1 previous similar message [ 3435.712927] LustreError: 132525:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3435.717973] LustreError: 132525:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3435.814019] Lustre: server umount lustre-MDT0000 complete [ 3440.070495] LustreError: 129175:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346399 with bad export cookie 9242693896276303470 [ 3440.085634] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3440.089556] LustreError: 129175:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3440.261158] LustreError: 132729:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3440.263303] LustreError: 132729:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3440.519652] Lustre: server umount lustre-MDT0001 complete [ 3451.865505] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3454.486302] Key type lgssc unregistered [ 3454.775137] LNet: 133261:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3454.795751] LNetError: 133261:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3454.815750] LNet: Removed LNI 192.168.201.158@tcp [ 3455.492197] Key type .llcrypt unregistered [ 3455.498053] Key type ._llcrypt unregistered [ 3477.347903] Key type ._llcrypt registered [ 3477.350034] Key type .llcrypt registered [ 3477.470200] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3478.274550] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3478.299897] alg: No test for adler32 (adler32-zlib) [ 3479.354335] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 3479.518253] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3481.231168] Key type lgssc registered [ 3482.299358] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3490.241599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3490.276939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3491.666652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3495.366550] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3501.043246] LustreError: 134655:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3501.053482] LustreError: 134655:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3502.704558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3503.080738] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3506.635862] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3511.717950] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3514.730683] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3520.710372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3521.053185] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3526.210809] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3528.248906] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3530.901243] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3536.441880] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3540.486667] Lustre: 136904:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3542.846857] Lustre: 136952:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3544.230912] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3545.061118] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3548.277233] Lustre: 137098:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3549.004326] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3549.756949] Lustre: 137196:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3553.760298] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3553.763575] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3553.773152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3556.834506] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3556.840303] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3556.853808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3556.869578] Lustre: Skipped 1 previous similar message [ 3558.999132] LustreError: 137293:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3559.112794] Lustre: server umount lustre-OST0000 complete [ 3564.512880] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3564.533389] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3564.559847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3567.072679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3567.074280] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3568.816207] LustreError: 137502:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3568.819087] LustreError: 137502:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3568.925706] Lustre: server umount lustre-MDT0000 complete [ 3572.199671] LustreError: 135367:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3572.219217] LustreError: 135367:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3572.241763] LustreError: 134637:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346531 with bad export cookie 18387939725931294635 [ 3572.246870] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3572.255633] LustreError: 134637:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3572.387694] LustreError: 137705:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3572.390845] LustreError: 137705:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3572.499314] Lustre: server umount lustre-MDT0001 complete [ 3581.141828] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3583.478815] Key type lgssc unregistered [ 3583.733592] LNet: 138238:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3583.738435] LNetError: 138238:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3583.751197] LNet: Removed LNI 192.168.201.158@tcp [ 3584.650644] Key type .llcrypt unregistered [ 3584.652849] Key type ._llcrypt unregistered [ 3608.208371] Key type ._llcrypt registered [ 3608.210731] Key type .llcrypt registered [ 3608.319209] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3609.307468] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3609.331756] alg: No test for adler32 (adler32-zlib) [ 3610.371812] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 3610.567776] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3612.223241] Key type lgssc registered [ 3613.575817] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3620.402372] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3620.422346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3621.935694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3625.922393] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3632.106310] LustreError: 139638:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3632.125234] LustreError: 139638:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3635.296442] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3635.641909] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3639.355197] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3644.966736] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3648.443450] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3655.239099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3655.609575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3660.718407] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3661.000712] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3665.839682] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3666.863871] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3672.006457] Lustre: *** cfs_fail_loc=159, val=0*** [ 3688.650891] Lustre: lustre-MDT0000: Client 768d2d6a-92c6-4b5e-952d-e6f83d6563be (at 192.168.201.58@tcp) reconnecting [ 3692.512733] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3692.515589] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3692.519170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3692.523425] Lustre: Skipped 1 previous similar message [ 3692.531625] Lustre: Skipped 1 previous similar message [ 3697.599995] LustreError: 142013:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3697.634046] LustreError: 141806:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3697.752762] Lustre: server umount lustre-OST0000 complete [ 3702.242770] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3702.249840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3702.277774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3707.685923] LustreError: 142214:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3707.691749] LustreError: 142214:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3707.758216] Lustre: server umount lustre-MDT0000 complete [ 3707.877094] LustreError: 140855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3707.889312] LustreError: 140855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3710.451324] LustreError: 142216:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346669 with bad export cookie 14173097037576512470 [ 3710.458499] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3710.466052] LustreError: 142216:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3710.581182] LustreError: 142417:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3710.583313] LustreError: 142417:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3710.693722] Lustre: server umount lustre-MDT0001 complete [ 3717.903227] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3720.115308] Key type lgssc unregistered [ 3720.403249] LNet: 142950:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3720.408723] LNetError: 142950:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3720.429968] LNet: Removed LNI 192.168.201.158@tcp [ 3721.078524] Key type .llcrypt unregistered [ 3721.080610] Key type ._llcrypt unregistered [ 3740.177676] Key type ._llcrypt registered [ 3740.183886] Key type .llcrypt registered [ 3740.285134] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3740.870077] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3740.967130] alg: No test for adler32 (adler32-zlib) [ 3741.980615] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 3742.169933] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3743.825342] Key type lgssc registered [ 3744.725672] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3750.806625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3750.820600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3752.181364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3755.079198] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3761.856175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3762.230345] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3765.104691] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3768.710738] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3771.402845] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3776.509132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3776.863369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3777.908142] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3782.127713] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3787.124580] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3787.581683] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3806.681563] Lustre: lustre-MDT0001: evicting 1ded1a9c-544b-4266-aeec-a37de753f02f (at 192.168.201.58@tcp) by administrative request [ 3806.695545] Lustre: Skipped 1 previous similar message [ 3834.337142] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3834.353126] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3834.361706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3838.437595] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3838.438141] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3838.438350] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3838.453201] Lustre: Skipped 1 previous similar message [ 3843.559281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3843.562033] Lustre: Skipped 1 previous similar message [ 3848.159191] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3848.209195] LustreError: 147106:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3848.373690] Lustre: server umount lustre-OST0000 complete [ 3853.792979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3853.819394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3853.828256] Lustre: Skipped 1 previous similar message [ 3858.912591] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3858.926120] Lustre: Skipped 1 previous similar message [ 3866.080757] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3866.205454] LustreError: 147310:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3866.216692] LustreError: 147310:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3866.374300] Lustre: server umount lustre-MDT0000 complete [ 3868.135231] LustreError: 144342:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3868.142559] LustreError: 144342:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3869.833307] LustreError: 144305:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346829 with bad export cookie 15430715481044355520 [ 3869.836668] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3869.859133] LustreError: 144305:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3869.962853] LustreError: 147513:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3869.967302] LustreError: 147513:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3870.179984] Lustre: server umount lustre-MDT0001 complete [ 3878.907201] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 3880.931287] Key type lgssc unregistered [ 3881.132609] LNet: 148044:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3881.137055] LNetError: 148044:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3881.147951] LNet: Removed LNI 192.168.201.158@tcp [ 3881.744125] Key type .llcrypt unregistered [ 3881.745763] Key type ._llcrypt unregistered [ 3942.392104] Key type ._llcrypt registered [ 3942.393393] Key type .llcrypt registered [ 3942.476204] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 3943.184851] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3943.195915] alg: No test for adler32 (adler32-zlib) [ 3944.209160] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 3944.389058] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 3946.039254] Key type lgssc registered [ 3946.889584] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3953.667854] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3953.690770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3955.074014] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3958.529964] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3965.413750] LustreError: 150493:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3965.431247] LustreError: 150493:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3967.735346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3968.144243] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3971.638165] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3977.130304] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3980.717689] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3987.863863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3988.140301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3993.164795] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 3993.275705] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3997.751961] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4000.440801] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4008.928333] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4008.939965] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4008.961697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4010.976578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4010.979436] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4014.943104] LustreError: 152748:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4015.039678] Lustre: server umount lustre-OST0000 complete [ 4019.170376] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4019.180848] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4019.192237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4019.206852] Lustre: Skipped 1 previous similar message [ 4021.217190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4021.225805] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4024.248941] LustreError: 152950:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4024.288437] LustreError: 152950:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4024.349934] Lustre: server umount lustre-MDT0000 complete [ 4026.337463] LustreError: 152450:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4026.350650] LustreError: 152450:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4027.258108] LustreError: 150472:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763346986 with bad export cookie 10383843666233331608 [ 4027.263096] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4027.271397] LustreError: 150472:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4027.437362] LustreError: 153151:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4027.443325] LustreError: 153151:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4027.591674] Lustre: server umount lustre-MDT0001 complete [ 4035.079401] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4037.318312] Key type lgssc unregistered [ 4037.554640] LNet: 153682:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4037.561931] LNetError: 153682:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4038.631743] LNet: Removed LNI 192.168.201.158@tcp [ 4039.252546] Key type .llcrypt unregistered [ 4039.253834] Key type ._llcrypt unregistered [ 4058.049309] Key type ._llcrypt registered [ 4058.054128] Key type .llcrypt registered [ 4058.164315] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4059.230388] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4059.272273] alg: No test for adler32 (adler32-zlib) [ 4060.331619] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 4060.545781] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4062.249406] Key type lgssc registered [ 4063.296472] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4069.690709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4077.179829] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 21:37:14 (1763347034) [ 4093.724692] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 4102.634868] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4111.906159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4118.379801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4124.551041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4132.112174] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4140.285652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4140.360338] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4140.378177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4141.613855] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4141.644203] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4141.706710] Lustre: lustre-MDT0000: new disk, initializing [ 4141.773350] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4141.789790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4145.302512] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4156.063390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4156.198849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4156.358273] Lustre: 157887:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4156.421923] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4156.424782] Lustre: Skipped 1 previous similar message [ 4156.503154] Lustre: lustre-MDT0001: new disk, initializing [ 4156.638254] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4156.700512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4156.714821] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4160.107683] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4167.092098] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4170.390451] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4176.567552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4176.641148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4176.839831] Lustre: lustre-OST0000: new disk, initializing [ 4176.845298] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4176.936091] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4182.269642] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4184.645857] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4184.661522] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4184.749325] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4189.140560] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4200.419441] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4200.422296] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4200.426207] LustreError: Skipped 1 previous similar message [ 4200.440563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4200.447299] Lustre: Skipped 1 previous similar message [ 4203.467101] LustreError: 160471:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4203.543064] Lustre: server umount lustre-OST0000 complete [ 4208.104160] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4208.107729] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4208.117839] Lustre: Skipped 1 previous similar message [ 4208.122942] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4210.656552] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4210.659296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4210.687355] Lustre: Skipped 1 previous similar message [ 4213.618557] LustreError: 160673:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4213.629660] LustreError: 160673:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4213.796802] Lustre: server umount lustre-MDT0000 complete [ 4217.617932] LustreError: 157878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763347176 with bad export cookie 2404238252223374968 [ 4217.628385] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4217.634295] LustreError: 157878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4217.781622] LustreError: 160874:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4217.790921] LustreError: 160874:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4218.010113] Lustre: server umount lustre-MDT0001 complete [ 4228.807616] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 4231.640461] Key type lgssc unregistered [ 4231.956671] LNet: 161407:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4231.960839] LNetError: 161407:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4231.975972] LNet: Removed LNI 192.168.201.158@tcp [ 4232.886182] Key type .llcrypt unregistered [ 4232.888731] Key type ._llcrypt unregistered [ 4257.138558] Key type ._llcrypt registered [ 4257.147728] Key type .llcrypt registered [ 4257.271346] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 4258.644296] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4258.731502] alg: No test for adler32 (adler32-zlib) [ 4259.838799] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 4260.075850] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 4261.799185] Key type lgssc registered [ 4263.413210] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4273.346215] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4273.386800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4275.029168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4279.522947] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4285.410785] LustreError: 162805:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4285.442549] LustreError: 162805:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4289.443900] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4290.005261] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4294.219143] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4299.787605] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4303.611959] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4309.675706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4310.042623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4315.074902] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 4319.646423] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4320.301614] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4329.500536] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4329.530276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4329.554241] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4329.565730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.581471] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4330.555176] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4330.593409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4330.614374] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4330.626220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.638412] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4331.413822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.364623] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4332.367488] Lustre: Skipped 1 previous similar message [ 4332.379633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4332.386077] Lustre: Skipped 1 previous similar message [ 4332.388587] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4332.398191] Lustre: Skipped 1 previous similar message [ 4332.401257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.408287] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4332.410664] Lustre: Skipped 1 previous similar message [ 4333.442902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.419670] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4334.424459] Lustre: Skipped 1 previous similar message [ 4334.447255] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4334.454724] Lustre: Skipped 1 previous similar message [ 4334.467813] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4334.480562] Lustre: Skipped 1 previous similar message [ 4334.486408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.500102] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4334.513424] Lustre: Skipped 1 previous similar message [ 4335.458789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.407203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.201189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.112271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.115493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4339.123769] Lustre: Skipped 4 previous similar messages [ 4339.141405] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4339.150366] Lustre: Skipped 4 previous similar messages [ 4339.156650] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4339.162375] Lustre: Skipped 4 previous similar messages [ 4339.166760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.183745] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4339.202817] Lustre: Skipped 4 previous similar messages [ 4339.906497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.791430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.689961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.397190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.069860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.837195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.561615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.217284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.128908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.011974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.843954] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4347.855255] Lustre: Skipped 10 previous similar messages [ 4347.883981] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4347.890271] Lustre: Skipped 10 previous similar messages [ 4347.894247] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4347.898223] Lustre: Skipped 10 previous similar messages [ 4347.901886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.917571] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4347.925497] Lustre: Skipped 10 previous similar messages [ 4348.780334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.598387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.339758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.047726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.857360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.704694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.514085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.271890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.084243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.866739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.672331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.539088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.330384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.128573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.965057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.846511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.638660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.464203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.368217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.215591] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4364.218340] Lustre: Skipped 19 previous similar messages [ 4364.228693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4364.235397] Lustre: Skipped 19 previous similar messages [ 4364.238631] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4364.248262] Lustre: Skipped 19 previous similar messages [ 4364.253649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.268229] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4364.280207] Lustre: Skipped 19 previous similar messages [ 4365.283169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.080863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.897060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.827134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.728463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.400651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.193383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.977945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.865719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.696607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.882199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.891716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.632022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.471385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.387146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.356836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.612595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.497377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.836600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.074566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.551163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.546569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.782490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.777487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.768924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.473580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.386911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.219454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.284203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.275467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.088068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.984550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.793390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.796609] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4396.802191] Lustre: Skipped 33 previous similar messages [ 4396.822140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4396.831217] Lustre: Skipped 33 previous similar messages [ 4396.837336] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4396.841606] Lustre: Skipped 33 previous similar messages [ 4396.850146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.858306] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4396.871209] Lustre: Skipped 33 previous similar messages [ 4397.860323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.784429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.672314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.789207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.888556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.885955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.016986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.095878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.096532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.082389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.868368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.968476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.925268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.993301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.140353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.001021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.118516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.222982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.119865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.227708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.225293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.424641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.515311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.714428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.742792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.803392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.811604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.776976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.977674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.043994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.129331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.258704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.393093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.570407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.601579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.585343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.666376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.710672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.728282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.470320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.430416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.535364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.308440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.213708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.268619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.273534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.226584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.120852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.087845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.118118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.150325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.005849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.166509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.085205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.113370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.095562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.170143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.055523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.977463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.912700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.026971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.929360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.998197] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4461.006154] Lustre: Skipped 62 previous similar messages [ 4461.027581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4461.034608] Lustre: Skipped 62 previous similar messages [ 4461.041091] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4461.043595] Lustre: Skipped 62 previous similar messages [ 4461.045581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.062104] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4461.077449] Lustre: Skipped 62 previous similar messages [ 4462.017954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.168647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.152540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.124054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.177335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.308378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.433284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.576284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.784188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.625754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.694525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.642208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.450666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.200848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.175645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.163137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.005373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.937507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.026890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.041428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.977797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.856764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.768957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.558600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.431406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.275562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.201972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.100457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.868826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.667104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.595361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.467661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.378730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.328041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.176764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.014679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.882406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.891257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.881515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.749400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.538094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.400717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.420318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.441777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.600236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.678366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.761854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.823738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.747428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.782438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.693701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.705083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.670189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.750331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.865397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.994535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.903907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.925628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.946161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.726202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.512677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.624508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.698351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.650087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.590268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.501610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.515309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.467168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.431836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.308409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.172836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.964571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.776726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.704645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.616826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.432355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.434329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.401141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.264534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.451076] LustreError: 162825:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4537.468138] LustreError: 162825:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4537.486249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.507365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.486861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.379829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.413483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.435545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.266059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.371940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.292260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.231976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.278022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.198950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.185723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.173419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.013457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.825836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.659848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.585335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.515228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.359585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.286972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.197584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.059730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.973806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.857492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.716143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.474390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.220275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.994567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.809599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.571244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.325287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.266962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.034538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.935211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.735809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.541268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.323725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.116222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.977492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.948443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.908362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.729098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.666293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.578331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.474672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.463473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.310191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.165927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.111654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.867298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.798576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.656244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.396182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.296854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.146567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.990784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.842344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.646897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.480149] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4589.482903] Lustre: Skipped 138 previous similar messages [ 4589.506855] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4589.531181] Lustre: Skipped 138 previous similar messages [ 4589.534932] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4589.537952] Lustre: Skipped 138 previous similar messages [ 4589.554243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.585331] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4589.595247] Lustre: Skipped 138 previous similar messages [ 4590.587555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.481492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.550974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.514845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.525061] LustreError: 162825:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4594.533403] LustreError: 162825:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4594.539470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.419655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.545419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.348801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.242896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.160203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.093473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.918835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.902333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.051278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.995768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.019245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.913926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.882708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.879341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.920430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.041480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.294096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.361877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.475252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.617022] LustreError: 162825:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4614.635100] LustreError: 162825:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4614.637628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.857151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.076536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.143282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.416346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.763544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.828850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.915183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.996283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.126137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.879313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.964538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.900712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.892720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.768437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.749995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.707541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.578430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.524753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.701963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.661259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.845505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.801476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.761917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.984872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.018774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.016150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.366965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.778208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.074592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.519640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.006170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.982769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.963637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.912608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.968984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.960878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.381985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.801261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.127043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.724844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.742672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.443739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.604324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.602751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.859607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.307919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.449203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.663106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.932585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.051816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.142726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.097882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.112994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.042982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.083375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.165182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.103151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.251842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.077342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.182819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.202551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.413714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.422517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.626119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.773671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.673414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.772916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.033225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.061205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.905691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.826903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.714782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.517075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.447557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.570038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.527277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.450443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.331340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.346264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.222781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.245397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.087654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.144600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.987836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.902121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.876776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.780774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.799055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.672413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.510036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.401413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.252675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.149222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.159064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.181175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.113135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.961717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.069912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.913156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.913672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.817246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.596637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.392691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.290260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.162355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.059799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.108868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.062382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.004083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.889664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.641610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.514898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.466610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.274764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.114501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.069884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.851329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.572563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.528532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.651145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.653413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.797522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.818968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.797491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.741563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.802440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.669085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.820640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.883168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.927209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.236717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.386741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.206687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.239132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.376909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.462089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.485299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.620266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.679861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.839463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.720939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.013545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.044576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.888277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.837370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.761058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.804392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.783321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.934475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.728235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.823228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.870222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.946517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.807822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.929694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.883445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.739697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.569883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.437029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.217128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.001822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.790607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.634986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.463493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.330881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.204561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.055183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.966750] LustreError: 162825:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4784.979312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.980364] LustreError: 162825:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4785.871898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.786995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.767417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.853173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.780739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.730577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.745044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.833604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.676266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.558625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.382338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.272272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.147661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.188173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.128775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.097027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.238483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.230075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.070280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.917119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.766111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.568170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.548326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.868088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.969197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.036732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.255489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.195408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.081801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.108820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.144383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.024535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.968375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.110871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.421620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.293367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.233994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.240894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.157753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.053709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.070616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.074285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.938533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.901915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.832248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.629557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.511123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.251758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.985252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.824990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.758942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.862205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.856653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.789484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.696204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.545416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.394637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.253727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.064463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.121552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.935645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.068991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.041327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.324526] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4846.326641] Lustre: Skipped 256 previous similar messages [ 4846.368384] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4846.373795] Lustre: Skipped 256 previous similar messages [ 4846.376616] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4846.379405] Lustre: Skipped 256 previous similar messages [ 4846.390951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.410535] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4846.412589] Lustre: Skipped 256 previous similar messages [ 4847.604736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.821260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.159046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.442229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.455629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.321603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.313169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.429708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.491960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.421381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.556698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.644858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.799253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.830314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.948356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.093321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.096782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.264819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.549430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.642458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.693655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.761908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.735260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.727389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.780926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.794419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.959979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.899277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.052184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.980487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.098141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.100133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.395693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.353819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.537287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.651032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.731332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.586991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.633975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.569669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.761844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.847722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.777985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.999744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.011431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.244817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.380723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.387568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.483388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.631807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.582501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.642828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.789698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.946818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.935945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.948339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.872984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.855393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.725696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.775140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.769874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.673260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.539312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.452330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.314781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.209218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.189228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.204590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.158483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.183463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.190583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.179146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.140048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.170924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.085130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.027145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.936319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.105042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.109379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.984375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.751642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.658264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.645710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.534785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.655274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.754794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.824688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.013368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.222654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.130975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.185206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.123864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.004722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.946942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.049654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.157731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.182280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.045486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.998945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.956455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.886851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.887815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.703434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.653645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.482945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.361309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.236207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.087074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.924830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.631964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.292381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.158226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.019615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.916768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.675267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.455062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.240167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.077352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.912168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.665920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.398819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.336608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.223806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.144829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.216445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.273980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.154637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.066223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.924949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.818066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.647884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.487800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.259141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.200975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.140900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.029643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.946269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.850578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.770150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.745547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.704510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.746321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.675805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.625912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.672534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.745407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.640773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.511278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.388270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.506494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.499640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.353674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.151897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.056069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.925415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.790655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.606129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.446867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.334831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.431309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.549848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.629271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.567258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.351859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.357045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.309067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.405962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.363133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.261266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.296771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.224110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.291435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.191435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.196370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.105837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.266361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.296928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.193951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.094715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.999349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.895728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.607429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.599297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.541628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.430648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.419271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.434409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.350649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.367863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.334997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.330451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.238784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.187799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.180826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.063686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.055900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.982721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.791147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.704205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.485110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.194230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.905740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.718706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.611263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.355510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.060342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.732083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.505215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.255802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.984240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.755179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.530435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.456084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.396397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.121956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.877544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.605438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.263637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.177051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.031881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.833414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.676633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.502992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.354578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.280301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.160917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.042546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.020543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.001418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.885547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.648406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.457527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.347643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.149467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.063756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.941266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.704651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.582878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.566888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.432515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.440973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.218447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.115282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.909421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.834873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.696222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.656477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.587416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.577733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.608556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.537184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.620745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.616437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.547593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.544166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.853714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.078554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.313170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.224393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.220241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.151262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.094033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.945838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.959947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.054963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.380818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.650326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.852536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.003437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.046275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.129268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.419780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.381587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.380421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.424662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.232423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.026378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.670150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.440748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.208638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.000106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.683606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.418544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.112557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.828173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.697278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.488310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.251581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.117524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.000204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.011090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.087366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.967247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.867835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.695505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.693863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.609955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.571369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.557678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.597170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.409542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.316471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.323400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.356636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.310702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.161045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.126034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.056729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.946853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.824279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.692569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.602797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.497516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.428588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.257672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.259471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.180477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.080863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.835696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.509421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.209359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.934420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.743429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.533191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.327683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.131864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.934135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.651906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.396707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.096618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.897279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.674446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.448467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.150473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.970958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.883379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.730745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.587876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.259447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.041262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.789459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.751427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.610377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.446783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.396495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.503413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.286703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.181807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.105889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.193534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.216130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.112989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.067226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.078265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.005899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.823928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.579902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.315374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.126036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.899665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.636602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.431531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.202953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.161489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.065079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.935250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.637641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.481601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.170746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.855221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.575037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.549473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.311287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.194728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.083680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.050907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.902953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.734318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.613707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.593407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.725163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.813258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.949624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.022637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.068546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.041416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.048680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.052093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.020658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.128898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.049647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.985760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.809183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.841415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.875632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.942581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.762494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.665488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.389402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.154381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.974077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.837948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.626615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.468409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.282971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.990718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.768298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.690029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.511439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.191655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.882657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.727285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.554624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.390538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.036025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.718784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.506927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.151564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.852370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.765522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.683585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.457525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.231587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.041646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.904730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.715360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.512953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.313045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.106115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.113552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.894950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.783181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.586100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.426048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.400084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.305448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.113728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.836903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.529658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.433799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.349989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.189546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.158940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.114723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.993269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.915213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.768238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.720262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.559814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.422628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.352996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.386521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.262910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.137672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.863250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.709804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.534887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.390044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.117118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.871227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.601730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.349280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.148917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.089095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.022268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.213754] LustreError: 164403:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5707.230231] LustreError: 162825:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6391.777316] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6391.782556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6391.795895] Lustre: Skipped 465 previous similar messages [ 6391.807426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6393.826126] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6393.844693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6395.882116] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6395.888793] Lustre: Skipped 1 previous similar message [ 6397.695305] LustreError: 216795:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6397.853779] Lustre: server umount lustre-OST0000 complete [ 6404.579375] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6404.587445] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6407.327810] LustreError: 216998:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6407.338719] LustreError: 216998:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6407.564841] Lustre: server umount lustre-MDT0000 complete [ 6410.905966] LustreError: 162787:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763349370 with bad export cookie 12851375953931679835 [ 6410.920731] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6411.070403] LustreError: 217199:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6411.073484] LustreError: 217199:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6411.226835] Lustre: server umount lustre-MDT0001 complete [ 6419.860842] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 6423.282801] Key type lgssc unregistered [ 6423.524708] LNet: 217732:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6423.542891] LNetError: 217732:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6424.556454] LNet: Removed LNI 192.168.201.158@tcp [ 6425.473545] Key type .llcrypt unregistered [ 6425.475129] Key type ._llcrypt unregistered [ 6448.168704] Key type ._llcrypt registered [ 6448.170358] Key type .llcrypt registered [ 6448.253531] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6448.883832] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6448.896807] alg: No test for adler32 (adler32-zlib) [ 6449.842720] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 6450.007304] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 6451.655707] Key type lgssc registered [ 6452.487594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6459.513487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6459.532452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6461.060439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6465.760669] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6471.653293] LustreError: 219147:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6471.680342] LustreError: 219147:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6476.288775] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6476.826416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6480.486700] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6486.042853] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6489.887544] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6497.296168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6497.732864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6503.448609] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6505.906124] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 6508.780600] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6514.983702] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6516.896849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6516.905797] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6516.925767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.936901] LustreError: 219167:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6516.940083] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6521.316557] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6521.320154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6521.321927] Lustre: Skipped 1 previous similar message [ 6521.327136] Lustre: Skipped 1 previous similar message [ 6522.052636] Lustre: lustre-OST0000: Not available for connect from 192.168.201.58@tcp (stopping) [ 6525.131969] LustreError: 221548:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6525.278667] Lustre: server umount lustre-OST0000 complete [ 6536.967156] LustreError: 219167:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6558.404165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6558.746139] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6561.836908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.862363] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.865153] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6561.901442] Lustre: Skipped 1 previous similar message [ 6564.119619] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6569.140029] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6570.691772] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6574.883131] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a5dc7d62800.ost_server_uuid 50 [ 6576.170237] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a5dc7d62800.ost_server_uuid in FULL state after 0 sec [ 6581.729855] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6581.753062] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6581.771562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6585.584487] LustreError: 222867:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6585.599917] LustreError: 222867:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6585.726257] Lustre: server umount lustre-OST0000 complete [ 6589.409592] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6589.412922] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6589.434798] Lustre: Skipped 1 previous similar message [ 6589.443858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6589.446676] Lustre: Skipped 2 previous similar messages [ 6595.172561] LustreError: 223068:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6595.175646] LustreError: 223068:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6595.228120] Lustre: server umount lustre-MDT0000 complete [ 6597.606772] LustreError: 219147:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6597.614166] LustreError: 219147:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6599.311566] LustreError: 219127:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763349558 with bad export cookie 3120148534610984157 [ 6599.317474] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6599.321390] LustreError: 219127:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6599.443382] LustreError: 223269:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6599.445534] LustreError: 223269:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6599.644296] Lustre: server umount lustre-MDT0001 complete [ 6609.611636] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 6611.920445] Key type lgssc unregistered [ 6612.230777] LNet: 223801:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6612.240786] LNetError: 223801:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6612.282547] LNet: Removed LNI 192.168.201.158@tcp [ 6612.984496] Key type .llcrypt unregistered [ 6612.986296] Key type ._llcrypt unregistered [ 6641.410259] Key type ._llcrypt registered [ 6641.412900] Key type .llcrypt registered [ 6641.509065] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6642.241717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6642.273067] alg: No test for adler32 (adler32-zlib) [ 6643.270599] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 6643.459327] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 6645.127403] Key type lgssc registered [ 6646.211704] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6654.588751] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6654.605204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6656.032984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6659.415119] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6666.213732] LustreError: 225515:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6666.236523] LustreError: 225515:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6667.388942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6667.776347] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6671.138318] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6673.806101] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6680.038699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6680.488200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6681.549961] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 6685.014651] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6692.343845] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6692.395402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6692.546240] Lustre: lustre-OST0001: new disk, initializing [ 6692.553252] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6692.653659] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6697.599811] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6702.688906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6702.698401] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6702.817794] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6706.971114] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6712.218854] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6733.798262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6733.801936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6733.810314] Lustre: Skipped 2 previous similar messages [ 6733.817568] Lustre: Skipped 3 previous similar messages [ 6736.053793] LustreError: 228875:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 6736.152514] Lustre: server umount lustre-MDT0000 complete [ 6738.802517] LustreError: 227782:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763349698 with bad export cookie 962833496326962702 [ 6738.806894] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6738.807800] LustreError: 227782:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6738.913245] LustreError: 229076:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6738.915699] LustreError: 229076:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6738.922145] LustreError: 225509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6738.922961] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6738.936289] LustreError: 225509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6738.942757] Lustre: Skipped 1 previous similar message [ 6738.947610] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6739.044259] Lustre: server umount lustre-MDT0001 complete [ 6748.229534] LustreError: 229277:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6748.235658] LustreError: 229277:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6748.283931] Lustre: server umount lustre-OST0000 complete [ 6751.362670] LustreError: 229479:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6751.366141] LustreError: 229479:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6751.564158] Lustre: server umount lustre-OST0001 complete [ 6754.343502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6754.436936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6755.302273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6755.417452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6756.358049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6756.416509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6757.256714] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6757.317481] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6768.870345] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6776.352462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6776.668397] LustreError: 230886:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6776.769169] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6779.793896] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6781.921194] LustreError: 230886:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6786.020443] LustreError: 230886:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6787.369444] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6787.748505] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6790.677980] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6792.722610] Lustre: 231995:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6797.048215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6797.265054] LustreError: 232352:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6800.778911] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6801.332587] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 6807.264429] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6807.441081] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6807.450543] Lustre: Skipped 1 previous similar message [ 6811.654604] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6818.984743] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6822.585903] Lustre: 233842:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6835.349732] LustreError: 234151:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6835.352761] LustreError: 234151:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6835.425812] Lustre: server umount mylustre-MDT0000 complete [ 6836.198428] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6836.204412] LustreError: 230881:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6836.205270] Lustre: Skipped 1 previous similar message [ 6836.214435] LustreError: 230881:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 6838.209112] LustreError: 230860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763349797 with bad export cookie 962833496326965516 [ 6838.212306] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6838.214294] LustreError: 230860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6840.599228] Lustre: server umount mylustre-MDT0001 complete [ 6854.282141] LustreError: 234553:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6854.288668] LustreError: 234553:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6854.323896] Lustre: server umount mylustre-OST0000 complete [ 6857.251248] Lustre: 224778:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763349800/real 1763349800] req@ffff9bb7e892c380 x1849006072416768/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763349816 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6857.266423] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6857.272656] Lustre: Skipped 2 previous similar messages [ 6857.439692] Lustre: server umount mylustre-OST0001 complete [ 6859.854252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6859.944821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6860.803531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6860.911926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6861.704682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6861.781841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6862.601493] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6862.719726] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6872.815916] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 6880.738802] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6881.107804] LustreError: 236163:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6881.235978] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6886.031615] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6895.627799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6900.723912] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6904.553139] Lustre: 237272:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6912.210601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6918.779843] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6927.358703] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 6927.508853] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6933.407968] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 6940.098624] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6943.557090] Lustre: 239120:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6957.536415] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6957.539536] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6957.577139] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6957.579898] Lustre: Skipped 1 previous similar message [ 6961.073058] LustreError: 239369:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6961.076091] LustreError: 239369:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6961.131878] Lustre: server umount tfs-MDT0000 complete [ 6963.686107] LustreError: 236162:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6963.698096] LustreError: 236162:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 10 previous similar messages [ 6964.082890] LustreError: 236139:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763349923 with bad export cookie 962833496326967182 [ 6964.098134] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6964.331737] Lustre: server umount tfs-MDT0001 complete [ 6977.086119] Lustre: server umount tfs-OST0000 complete [ 6990.104716] Lustre: server umount tfs-OST0001 complete [ 6993.080340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6993.157660] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6994.270938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6994.341022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6995.549241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6995.660830] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6996.688412] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6996.770661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7008.062633] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7015.463789] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7015.934532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7015.937894] Lustre: Skipped 3 previous similar messages [ 7018.959514] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7025.919623] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7029.638849] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7032.042734] Lustre: 242496:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7037.456607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7040.891583] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 7043.043370] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7050.041799] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7056.005196] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7063.746449] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7067.642142] Lustre: 244346:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7080.209901] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 22:27:17 (1763350037) [ 7097.313773] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7097.317830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7097.334095] Lustre: Skipped 5 previous similar messages [ 7097.360636] Lustre: Skipped 7 previous similar messages [ 7101.387399] LustreError: 244714:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7101.390127] LustreError: 244714:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7101.561411] Lustre: server umount lustre-MDT0000 complete [ 7102.453324] LustreError: 241382:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7102.472064] LustreError: 241382:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 7105.543864] LustreError: 241363:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763350064 with bad export cookie 962833496326968617 [ 7105.553870] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7105.554831] LustreError: 241363:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 7107.561828] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7107.564536] Lustre: Skipped 1 previous similar message [ 7111.949197] Lustre: server umount lustre-MDT0001 complete [ 7121.079512] Lustre: server umount lustre-OST0000 complete [ 7130.304803] Lustre: server umount lustre-OST0001 complete [ 7134.906479] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 7143.493237] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7152.229791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7158.204092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7162.931913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7167.659635] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7173.152113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7173.211189] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7173.211189] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7173.211189] [ 7173.230050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7173.422632] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7173.449531] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7173.533832] Lustre: lustre-MDT0000: new disk, initializing [ 7173.622656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7173.625393] Lustre: Skipped 3 previous similar messages [ 7173.645946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7176.516206] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7185.496472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7185.567661] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7185.646279] Lustre: 247505:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7185.741146] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7185.743710] Lustre: Skipped 1 previous similar message [ 7185.803956] Lustre: lustre-MDT0001: new disk, initializing [ 7185.963753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7185.977340] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7189.569102] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7198.488821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7198.542149] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7198.758720] Lustre: lustre-OST0000: new disk, initializing [ 7198.761765] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7200.678855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7200.689817] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7200.786947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7203.064806] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7211.900068] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7211.984470] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7212.112874] Lustre: lustre-OST0001: new disk, initializing [ 7212.117134] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7213.320554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7213.325613] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7213.498149] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7217.720592] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7227.507198] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 22:29:45 (1763350185) [ 7239.755167] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 22:29:57 (1763350197) [ 7254.502286] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7254.507568] Lustre: Skipped 3 previous similar messages [ 7254.518492] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7254.521062] Lustre: Skipped 3 previous similar messages [ 7268.319347] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7268.368187] LustreError: 250928:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7268.380609] LustreError: 250928:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7268.459148] Lustre: server umount lustre-OST0000 complete [ 7272.935327] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7272.946321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7272.951783] Lustre: Skipped 4 previous similar messages [ 7278.106680] Lustre: server umount lustre-MDT0000 complete [ 7282.025934] LustreError: 247497:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763350241 with bad export cookie 962833496326969954 [ 7282.030285] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7282.032155] LustreError: 247497:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7288.313942] Lustre: server umount lustre-MDT0001 complete [ 7296.137593] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 7313.528911] Lustre: server umount lustre-OST0001 complete [ 7318.865559] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 7331.410133] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7342.374038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7349.425103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7356.506872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7362.441748] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7368.207434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7368.255513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7368.431326] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7368.458356] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7368.545668] Lustre: lustre-MDT0000: new disk, initializing [ 7368.641159] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7372.104952] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7380.679681] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7380.749799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7380.838391] Lustre: 254408:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7381.090733] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7383.803294] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7390.249500] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7393.235952] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7398.747784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7398.829633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7399.073396] Lustre: lustre-OST0000: new disk, initializing [ 7399.075268] Lustre: Skipped 1 previous similar message [ 7399.083530] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7399.086522] Lustre: Skipped 2 previous similar messages [ 7400.576280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7400.591214] Lustre: Skipped 1 previous similar message [ 7400.596754] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7400.716433] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7403.621288] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7409.720469] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7416.296352] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7416.305480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7416.317942] Lustre: Skipped 4 previous similar messages [ 7416.327536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7416.334485] Lustre: Skipped 6 previous similar messages [ 7420.795308] LustreError: 256849:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7420.805688] LustreError: 256849:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7420.908768] Lustre: server umount lustre-OST0000 complete [ 7430.483349] Lustre: server umount lustre-MDT0000 complete [ 7431.648915] LustreError: 254417:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7431.660432] LustreError: 254417:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 7433.165667] LustreError: 254400:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763350392 with bad export cookie 962833496326971424 [ 7433.170313] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7433.171303] LustreError: 254400:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7433.458442] Lustre: server umount lustre-MDT0001 complete [ 7441.960877] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 7444.329314] Key type lgssc unregistered [ 7444.666684] LNet: 257784:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7444.678719] LNetError: 257784:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7445.735589] LNet: Removed LNI 192.168.201.158@tcp [ 7446.425170] Key type .llcrypt unregistered [ 7446.428306] Key type ._llcrypt unregistered [ 7471.093048] Key type ._llcrypt registered [ 7471.095380] Key type .llcrypt registered [ 7471.231254] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7472.719040] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7472.734477] alg: No test for adler32 (adler32-zlib) [ 7473.870906] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 7474.107737] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 7475.727188] Key type lgssc registered [ 7477.119818] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7485.205723] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7485.224830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7487.017346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7491.129256] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7497.186355] LustreError: 259227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7497.213656] LustreError: 259227:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7499.432066] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7499.680171] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7503.231467] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7508.073497] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7511.595073] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7517.437958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7517.830584] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7523.129911] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7527.752758] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7528.662805] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7532.258332] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7532.261951] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7536.312085] LustreError: 261520:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7536.401344] Lustre: server umount lustre-OST0000 complete [ 7540.704575] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7540.716654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7540.732443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7540.737345] Lustre: Skipped 1 previous similar message [ 7542.756129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7542.757103] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7545.540712] LustreError: 261726:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7545.545267] LustreError: 261726:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7545.650095] Lustre: server umount lustre-MDT0000 complete [ 7547.877695] LustreError: 259221:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7547.910963] LustreError: 259221:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7549.111293] LustreError: 259208:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763350508 with bad export cookie 7048099361357997890 [ 7549.115909] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7549.124041] LustreError: 259208:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7549.299542] LustreError: 261930:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7549.308798] LustreError: 261930:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7549.477779] Lustre: server umount lustre-MDT0001 complete [ 7558.116536] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 7560.415055] Key type lgssc unregistered [ 7560.701652] LNet: 262462:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7560.710360] LNetError: 262462:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7560.724074] LNet: Removed LNI 192.168.201.158@tcp [ 7561.347270] Key type .llcrypt unregistered [ 7561.348699] Key type ._llcrypt unregistered [ 7573.433591] Key type ._llcrypt registered [ 7573.436182] Key type .llcrypt registered [ 7573.516839] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7574.293476] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7574.304986] alg: No test for adler32 (adler32-zlib) [ 7575.253890] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 7575.414528] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 7577.079204] Key type lgssc registered [ 7578.118816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7585.505454] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7585.518561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7600.991529] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7606.111319] LustreError: 263616:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.158@tcp: failed processing log, type 4: rc = -110 [ 7632.735570] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7637.772325] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7642.959082] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7645.045067] LustreError: 264283:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 7645.214587] Lustre: server umount lustre-OST0000 complete [ 7659.800155] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 7662.550788] Key type lgssc unregistered [ 7662.800621] LNet: 265119:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7662.812251] LNetError: 265119:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7662.831165] LNet: Removed LNI 192.168.201.158@tcp [ 7663.728305] Key type .llcrypt unregistered [ 7663.732479] Key type ._llcrypt unregistered [ 7706.709651] Key type ._llcrypt registered [ 7706.716330] Key type .llcrypt registered [ 7706.840569] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7707.998897] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7708.048601] alg: No test for adler32 (adler32-zlib) [ 7709.140906] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 7709.336765] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 7710.999230] Key type lgssc registered [ 7712.053796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7723.653584] loop0: detected capacity change from 0 to 204800000 [ 7723.850645] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7723.858791] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7724.080574] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7724.122439] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7725.084435] loop0: detected capacity change from 0 to 204800000 [ 7725.242927] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7725.249857] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7725.573680] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7725.619444] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7726.607363] loop0: detected capacity change from 0 to 204800000 [ 7726.773256] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7726.783930] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7726.912556] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7727.005808] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7728.001524] loop0: detected capacity change from 0 to 204800000 [ 7728.223502] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7728.447260] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7729.472390] loop0: detected capacity change from 0 to 204800000 [ 7729.497914] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7733.016844] loop0: detected capacity change from 0 to 204800000 [ 7733.063404] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7736.372566] loop0: detected capacity change from 0 to 204800000 [ 7736.400333] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7739.816161] loop0: detected capacity change from 0 to 204800000 [ 7739.867084] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7743.345154] loop0: detected capacity change from 0 to 204800000 [ 7743.426852] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7743.457928] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7743.472203] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7743.526786] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7743.535135] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7743.545870] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7743.555539] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7743.564852] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7743.572705] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7743.582445] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7743.592989] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7743.600812] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7743.612410] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7743.624628] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7743.631258] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7743.637772] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7743.647526] Lustre: 268525:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7748.715769] LustreError: 268793:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7748.831037] Lustre: server umount MGS complete [ 7749.533610] loop0: detected capacity change from 0 to 204800000 [ 7749.616434] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7749.970145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7750.026922] Lustre: lustre-MDT0000: Aborting client recovery [ 7750.855498] loop1: detected capacity change from 0 to 204800000 [ 7751.009897] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7751.088837] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7751.123690] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7751.132692] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7751.140245] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7751.148785] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7751.155702] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7751.174241] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7751.186308] Lustre: 268962:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7751.207673] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7751.411250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7751.473874] Lustre: lustre-MDT0001: Aborting client recovery [ 7752.407648] loop2: detected capacity change from 0 to 204800000 [ 7752.513558] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7752.601205] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7752.638629] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7752.650499] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7752.659278] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7752.674969] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7752.688613] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7752.701420] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7752.714079] Lustre: 269044:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7752.825419] Lustre: Found index 0 for lustre-OST0000, updating log [ 7752.920665] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7752.945669] Lustre: lustre-OST0000: Aborting recovery [ 7753.886827] loop3: detected capacity change from 0 to 204800000 [ 7753.988150] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7754.051111] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7754.102138] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7754.119808] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7754.132367] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7754.146468] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7754.154586] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7754.164277] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7754.176949] Lustre: 269131:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7754.225297] Lustre: Found index 1 for lustre-OST0001, updating log [ 7754.384643] Lustre: lustre-OST0001: Aborting recovery [ 7759.873291] LustreError: 268905:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 7760.870557] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7760.872455] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7760.900844] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7760.902273] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7760.912056] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7760.912299] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7763.213071] LustreError: 268873:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4661us: rc = -2 [ 7763.730620] LustreError: 268873:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 947us: rc = -2 [ 7763.750649] LustreError: 268873:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 7777.256562] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7777.264102] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7777.286173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7781.345401] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7781.345945] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7781.382967] Lustre: Skipped 2 previous similar messages [ 7781.387448] Lustre: Skipped 2 previous similar messages [ 7782.681605] LustreError: 269491:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7782.684172] LustreError: 269491:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7782.887844] Lustre: server umount lustre-MDT0000 complete [ 7783.854761] LustreError: 268855:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763350743 with bad export cookie 10245230874532433794 [ 7783.863225] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7783.877613] LustreError: 268855:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7786.464073] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7786.468551] LustreError: 268870:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7786.507213] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7786.545534] LustreError: 268870:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7786.563326] Lustre: Skipped 1 previous similar message [ 7788.512118] LustreError: 268871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7788.515560] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7788.537197] LustreError: 268871:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7788.571982] Lustre: Skipped 1 previous similar message [ 7793.632527] LustreError: 269379:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7793.633885] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7793.659187] LustreError: 269379:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7793.691587] Lustre: Skipped 1 previous similar message [ 7798.263113] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7798.390363] LustreError: 269543:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 7798.399546] LustreError: 269543:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7798.723667] Lustre: server umount lustre-MDT0001 complete [ 7805.763330] LustreError: 269596:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7805.769096] LustreError: 269596:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7805.859312] Lustre: server umount lustre-OST0000 complete [ 7821.279168] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7821.341376] LustreError: 269647:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7821.343852] LustreError: 269647:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7821.507640] Lustre: server umount lustre-OST0001 complete [ 7830.135053] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 22:39:47 (1763350787) [ 7857.697289] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 7867.339746] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 7879.208496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7886.316835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7893.314813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7900.192776] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7906.426494] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7906.490895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7906.731842] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7906.766692] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7906.850858] Lustre: lustre-MDT0000: new disk, initializing [ 7906.963778] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7906.970256] Lustre: Skipped 1 previous similar message [ 7906.986705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7910.647846] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7922.077989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7922.148178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7922.215650] Lustre: 273199:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7922.269940] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7922.273713] Lustre: Skipped 1 previous similar message [ 7922.332658] Lustre: lustre-MDT0001: new disk, initializing [ 7922.461113] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7922.504924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7922.528786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7927.314780] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7937.250746] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7940.987336] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7948.821448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7948.888509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7949.220617] Lustre: lustre-OST0000: new disk, initializing [ 7949.224309] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7949.319328] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7955.606723] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 7957.564106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7957.587396] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7957.719911] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7963.073105] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7968.921583] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7978.274752] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7987.587940] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7999.368057] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8050.148704] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8050.152821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8050.164643] LustreError: Skipped 1 previous similar message [ 8050.165921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8050.171333] Lustre: Skipped 2 previous similar messages [ 8050.188249] Lustre: Skipped 3 previous similar messages [ 8052.431207] LustreError: 276798:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8052.452996] LustreError: 276798:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8052.626506] Lustre: server umount lustre-OST0000 complete [ 8060.400605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8060.413459] Lustre: Skipped 1 previous similar message [ 8062.777768] Lustre: server umount lustre-MDT0000 complete [ 8065.507758] LustreError: 273212:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8065.515560] LustreError: 273212:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8066.801388] LustreError: 273191:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763351026 with bad export cookie 10245230874532436496 [ 8066.802761] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8066.811111] LustreError: 273191:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8067.385212] Lustre: server umount lustre-MDT0001 complete [ 8077.513564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8081.982801] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8087.264697] LustreError: 278146:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8087.271474] LustreError: 278146:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 8087.358555] Lustre: server umount MGS complete [ 8098.446456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8098.919367] LustreError: 278768:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8098.960232] LustreError: 278780:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8099.048445] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8103.115962] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8109.539432] LustreError: 278768:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8109.553644] LustreError: 278768:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8110.852955] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8114.585167] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8119.641090] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8122.793851] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8129.569668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8135.043907] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8139.979601] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8169.955839] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8169.958522] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8169.961610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8169.967343] Lustre: Skipped 1 previous similar message [ 8169.974071] Lustre: Skipped 3 previous similar messages [ 8174.205076] LustreError: 281429:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8174.219495] LustreError: 281429:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8174.434556] Lustre: server umount lustre-OST0000 complete [ 8184.472977] Lustre: server umount lustre-MDT0000 complete [ 8185.313209] LustreError: 278764:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8185.336921] LustreError: 278764:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8188.676211] LustreError: 278749:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763351147 with bad export cookie 10245230874532438456 [ 8188.677036] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8188.688661] LustreError: 278749:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8189.070866] Lustre: server umount lustre-MDT0001 complete [ 8199.776955] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8202.819680] Key type lgssc unregistered [ 8203.184446] LNet: 282364:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8203.199903] LNetError: 282364:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8203.215732] LNet: Removed LNI 192.168.201.158@tcp [ 8203.986185] Key type .llcrypt unregistered [ 8203.992383] Key type ._llcrypt unregistered [ 8241.431966] Key type ._llcrypt registered [ 8241.433485] Key type .llcrypt registered [ 8241.549830] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 8253.252663] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8253.878298] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8253.951763] alg: No test for adler32 (adler32-zlib) [ 8254.985578] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 8255.166705] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 8256.823395] Key type lgssc registered [ 8257.823956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8266.903176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8273.600157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8279.582889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8286.981159] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8295.695656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8295.801489] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8295.832203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8297.349653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8297.420324] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8297.472495] Lustre: lustre-MDT0000: new disk, initializing [ 8297.575256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8297.633860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8302.619422] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8315.806851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8315.911150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8316.105994] Lustre: 286280:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8316.230708] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8316.244395] Lustre: Skipped 1 previous similar message [ 8316.496970] Lustre: lustre-MDT0001: new disk, initializing [ 8316.730611] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8316.827749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8316.840229] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8321.168465] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8328.099892] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8332.158553] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8339.561892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8339.662987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8340.109941] Lustre: lustre-OST0000: new disk, initializing [ 8340.120525] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8340.237652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8345.669793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8345.680270] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8345.850215] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8346.344972] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8354.471560] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8361.727152] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8372.372359] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8380.177241] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8422.882335] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8422.882581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8422.889506] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8422.917614] LustreError: Skipped 1 previous similar message [ 8422.935653] Lustre: Skipped 1 previous similar message [ 8428.007607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8428.019803] Lustre: Skipped 1 previous similar message [ 8428.610208] LustreError: 289847:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8428.755650] Lustre: server umount lustre-OST0000 complete [ 8434.656393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8434.660067] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8434.676607] Lustre: Skipped 1 previous similar message [ 8434.683828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8436.195674] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8438.638666] LustreError: 290048:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8438.642095] LustreError: 290048:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8438.690184] Lustre: server umount lustre-MDT0000 complete [ 8442.339561] LustreError: 289521:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763351401 with bad export cookie 14742233831569068914 [ 8442.343914] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8442.345696] LustreError: 289521:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8442.479595] LustreError: 290250:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8442.484675] LustreError: 290250:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8442.644030] Lustre: server umount lustre-MDT0001 complete [ 8451.153695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8455.264227] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8460.083796] LustreError: 291192:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8460.098264] LustreError: 291192:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8460.184478] Lustre: server umount MGS complete [ 8470.520796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8470.911658] LustreError: 291814:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8470.941286] LustreError: 291826:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8470.949840] LustreError: 291814:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8471.044423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8474.735813] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8476.135067] LustreError: 291813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8476.165562] LustreError: 291813:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8481.251676] LustreError: 291813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8483.014914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8483.362264] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8486.944155] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8492.175157] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8495.955484] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8502.268269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8502.731154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8508.243104] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8513.497095] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8543.716222] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8543.724839] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8543.749307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8543.752015] Lustre: Skipped 2 previous similar messages [ 8546.785810] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8548.609153] LustreError: 294472:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8548.614208] LustreError: 294472:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8548.740243] Lustre: server umount lustre-OST0000 complete [ 8554.981282] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8554.989808] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8554.999441] Lustre: Skipped 1 previous similar message [ 8555.006916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8555.009805] Lustre: Skipped 2 previous similar messages [ 8558.317534] LustreError: 294674:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8558.323064] LustreError: 294674:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8558.449619] Lustre: server umount lustre-MDT0000 complete [ 8562.144821] LustreError: 292527:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8562.165320] LustreError: 292527:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8562.362309] LustreError: 293434:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763351521 with bad export cookie 14742233831569070769 [ 8562.363274] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8562.371404] LustreError: 293434:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8562.677275] Lustre: server umount lustre-MDT0001 complete [ 8574.358274] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 8576.811263] Key type lgssc unregistered [ 8577.116926] LNet: 295407:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8577.122468] LNetError: 295407:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8578.149941] LNet: Removed LNI 192.168.201.158@tcp [ 8578.905713] Key type .llcrypt unregistered [ 8578.909077] Key type ._llcrypt unregistered [ 8597.801640] Key type ._llcrypt registered [ 8597.803157] Key type .llcrypt registered [ 8597.919605] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 8609.266113] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8610.267722] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8610.306398] alg: No test for adler32 (adler32-zlib) [ 8611.349962] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 8611.592176] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 8613.264722] Key type lgssc registered [ 8614.496401] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8622.749496] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8628.200334] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8638.435532] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 8648.127715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8655.235761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8655.298041] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8655.317532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8656.553943] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8656.597188] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8656.664523] Lustre: lustre-MDT0000: new disk, initializing [ 8656.833462] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8656.858264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8660.437352] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 8669.372765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8669.429899] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8669.624521] Lustre: lustre-OST0000: new disk, initializing [ 8669.628935] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8669.636371] Lustre: Skipped 1 previous similar message [ 8669.699596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8669.930775] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8669.934883] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8669.984368] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8674.984199] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9315.298379] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9315.305384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9316.496723] LustreError: 300414:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 9316.656549] Lustre: server umount lustre-MDT0000 complete [ 9318.694562] LustreError: 298580:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352277 with bad export cookie 17044556530949744703 [ 9318.697282] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9318.700501] LustreError: 298580:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9320.753859] LustreError: 300618:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9320.760162] LustreError: 300618:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9320.827231] Lustre: server umount lustre-OST0000 complete [ 9335.074217] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [ 9340.113424] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9344.976777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9348.502447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9352.282449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9356.143902] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9360.297806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9360.344461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9360.472789] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9360.495457] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9360.547236] Lustre: lustre-MDT0000: new disk, initializing [ 9360.589539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9360.601720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9362.683939] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9369.287243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9369.340372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9369.392737] Lustre: 303660:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9369.441505] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9369.445170] Lustre: Skipped 1 previous similar message [ 9369.492786] Lustre: lustre-MDT0001: new disk, initializing [ 9369.570516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9369.606971] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9369.615973] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9371.807581] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9376.725413] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9379.211662] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9383.741879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9383.787811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9383.957183] Lustre: lustre-OST0000: new disk, initializing [ 9383.963044] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9384.033378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9386.025372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9386.030480] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9386.106602] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9387.387113] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9391.860595] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9393.977251] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9394.084281] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9396.801856] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9396.935058] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9401.314338] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9401.317714] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9401.320655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9401.325621] Lustre: Skipped 1 previous similar message [ 9401.333451] Lustre: Skipped 1 previous similar message [ 9404.277419] LustreError: 306598:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9404.287123] LustreError: 306598:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9404.361562] Lustre: server umount lustre-OST0000 complete [ 9410.529179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9410.534290] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9410.542090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9412.621920] LustreError: 306800:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 9412.624934] LustreError: 306800:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9412.676190] Lustre: server umount lustre-MDT0000 complete [ 9415.027672] LustreError: 303652:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352374 with bad export cookie 17044556530952685550 [ 9415.029766] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9415.037069] LustreError: 303652:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9415.215747] Lustre: server umount lustre-MDT0001 complete [ 9417.405633] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9418.366231] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 23:06:16 (1763352376) [ 9422.145213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9422.442112] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9424.574692] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9427.937152] LustreError: 307591:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9427.952020] LustreError: 307591:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9429.455877] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9429.709731] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9431.763551] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9434.629873] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9436.431845] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9440.004884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9440.233330] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9443.321736] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9446.469457] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9450.635550] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9450.707368] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9450.821514] Lustre: lustre-OST0001: new disk, initializing [ 9450.824026] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9454.717219] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9456.149379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9456.165316] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9456.191749] LustreError: 310521:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9457.251532] LustreError: 310521:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9457.258287] LustreError: 310521:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9458.273317] LustreError: 310525:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9458.279820] LustreError: 310525:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9460.324390] LustreError: 310521:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9460.332159] LustreError: 310521:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9460.629332] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9460.767617] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9463.140335] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a5dd8d0c000.ost_server_uuid 50 [ 9464.105622] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a5dd8d0c000.ost_server_uuid in FULL state after 0 sec [ 9464.427600] LustreError: 310525:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9464.439566] LustreError: 310525:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9466.542424] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8a5dd8d0c000.ost_server_uuid 50 [ 9467.524663] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8a5dd8d0c000.ost_server_uuid in FULL state after 0 sec [ 9468.609386] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9481.706622] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9481.708361] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9481.712772] Lustre: Skipped 2 previous similar messages [ 9481.715896] Lustre: Skipped 2 previous similar messages [ 9486.822194] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9486.827232] Lustre: Skipped 3 previous similar messages [ 9495.007263] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9495.042951] LustreError: 311376:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 9495.046261] LustreError: 311376:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 9495.073851] Lustre: server umount lustre-OST0001 complete [ 9497.061502] LustreError: 309172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9497.068879] LustreError: 309172:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9498.593722] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9498.597187] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9498.609989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9498.614200] Lustre: Skipped 4 previous similar messages [ 9502.183831] LustreError: 309170:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9502.193783] LustreError: 309170:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9504.324157] LustreError: 311580:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9504.331233] LustreError: 311580:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9504.414607] Lustre: server umount lustre-OST0000 complete [ 9507.297989] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9507.304523] Lustre: Skipped 2 previous similar messages [ 9513.110924] Lustre: server umount lustre-MDT0000 complete [ 9515.389258] LustreError: 307554:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352474 with bad export cookie 17044556530952686432 [ 9515.389258] LustreError: 309192:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352474 with bad export cookie 17044556530952686432 [ 9515.391730] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9515.395280] LustreError: 307554:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9515.631421] Lustre: server umount lustre-MDT0001 complete [ 9521.539589] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 9523.111344] Key type lgssc unregistered [ 9523.276662] LNet: 312515:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9523.279497] LNetError: 312515:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9523.289831] LNet: Removed LNI 192.168.201.158@tcp [ 9523.797233] Key type .llcrypt unregistered [ 9523.799742] Key type ._llcrypt unregistered [ 9537.773118] Key type ._llcrypt registered [ 9537.776958] Key type .llcrypt registered [ 9537.842269] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9538.379419] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9538.409519] alg: No test for adler32 (adler32-zlib) [ 9539.327480] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 9539.457450] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 9541.083421] Key type lgssc registered [ 9541.776834] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9546.382103] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9546.400687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9547.710778] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9549.791655] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9552.703531] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9556.322326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9556.605549] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9558.897291] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9564.227630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9564.445785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9566.504046] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9566.528902] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9567.948799] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9571.439827] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9571.819404] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9572.851086] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9572.859500] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9572.863661] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9576.132644] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9576.283673] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9579.821633] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9581.554317] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9583.477318] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9587.629508] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9587.763973] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9683.939403] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9683.943876] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9683.944836] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9683.956476] Lustre: Skipped 1 previous similar message [ 9689.058145] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9689.072707] Lustre: Skipped 2 previous similar messages [ 9694.181374] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9694.183715] Lustre: Skipped 1 previous similar message [ 9695.200252] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9695.229239] LustreError: 317320:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 9695.287848] Lustre: server umount lustre-OST0001 complete [ 9699.298191] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9699.299340] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9699.301742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9699.311961] Lustre: Skipped 1 previous similar message [ 9699.314527] LustreError: 315448:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9704.419704] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9704.419925] LustreError: 316803:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9704.425643] Lustre: Skipped 2 previous similar messages [ 9704.431723] LustreError: 316803:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9704.699421] LustreError: 317525:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9704.706795] LustreError: 317525:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9704.799179] Lustre: server umount lustre-OST0000 complete [ 9709.540658] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9709.545733] Lustre: Skipped 1 previous similar message [ 9714.657569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9714.661196] Lustre: Skipped 3 previous similar messages [ 9721.311409] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9721.376187] LustreError: 317726:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9721.380643] LustreError: 317726:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9721.439736] Lustre: server umount lustre-MDT0000 complete [ 9723.841096] LustreError: 314714:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352683 with bad export cookie 546634112632855672 [ 9723.843064] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9723.851951] LustreError: 314714:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9723.936419] LustreError: 317928:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9723.938541] LustreError: 317928:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9724.072230] Lustre: server umount lustre-MDT0001 complete [ 9730.274243] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [ 9732.383784] Key type lgssc unregistered [ 9732.607751] LNet: 318459:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9732.612917] LNetError: 318459:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9732.622591] LNet: Removed LNI 192.168.201.158@tcp [ 9733.204313] Key type .llcrypt unregistered [ 9733.206414] Key type ._llcrypt unregistered [ 9762.882628] Key type ._llcrypt registered [ 9762.884314] Key type .llcrypt registered [ 9762.952969] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9763.421114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9763.500071] alg: No test for adler32 (adler32-zlib) [ 9764.420546] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [ 9764.571553] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [ 9766.199303] Key type lgssc registered [ 9766.903826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9771.956685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9771.966985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9773.245060] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9775.196947] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9779.636248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9779.876985] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9781.870309] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9783.568320] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9787.146333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9787.344765] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9790.424759] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9790.436524] LustreError: 322163:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9790.449551] LustreError: 322163:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9795.560403] LustreError: 322162:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9796.178248] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9796.354513] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9798.449309] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9798.449918] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9799.866194] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9806.774793] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9810.670601] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9824.228507] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9824.230080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9824.233485] Lustre: Skipped 2 previous similar messages [ 9824.242227] Lustre: Skipped 3 previous similar messages [ 9828.914654] LustreError: 324116:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9828.974502] Lustre: server umount lustre-MDT0000 complete [ 9829.347403] LustreError: 320700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9829.355645] LustreError: 320700:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9831.117698] LustreError: 323660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352790 with bad export cookie 12439304932124467969 [ 9831.118258] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9831.123813] LustreError: 323660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9831.223932] LustreError: 324317:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9831.226940] LustreError: 324317:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9831.316786] Lustre: server umount lustre-MDT0001 complete [ 9843.711517] LustreError: 324520:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9843.715528] LustreError: 324520:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9843.748549] Lustre: server umount lustre-OST0000 complete [ 9856.122347] LustreError: 324721:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9856.126499] LustreError: 324721:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9856.197864] Lustre: server umount lustre-OST0001 complete [ 9864.175550] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9870.100956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9870.374698] LustreError: 325898:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9870.457908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9872.927111] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9875.946264] LustreError: 325898:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9878.475564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9878.742470] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9881.305214] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9883.240642] Lustre: 327007:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9887.443488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9888.674837] LustreError: 327362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9890.760697] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9895.516788] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9895.660133] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9895.662909] Lustre: Skipped 1 previous similar message [ 9896.688518] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9896.692178] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9898.451407] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [ 9902.661660] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9904.397987] Lustre: 328853:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9912.291928] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9912.294141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9912.297825] Lustre: Skipped 4 previous similar messages [ 9912.300776] Lustre: Skipped 2 previous similar messages [ 9917.412470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9917.415643] LustreError: 328956:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9917.419503] Lustre: Skipped 4 previous similar messages [ 9917.424218] LustreError: 328956:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9917.489730] Lustre: server umount lustre-MDT0000 complete [ 9919.600247] LustreError: 325880:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763352878 with bad export cookie 12439304932124469663 [ 9919.601887] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9919.604775] LustreError: 325880:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9919.769934] Lustre: server umount lustre-MDT0001 complete [ 9932.796623] LustreError: 329359:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9932.798952] LustreError: 329359:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 9932.830878] Lustre: server umount lustre-OST0000 complete [ 9945.153077] Lustre: server umount lustre-OST0001 complete [ 9947.340573] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9948.400633] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9949.604639] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 23:15:07 (1763352907) [ 9962.379738] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [ 9966.959916] loop0: detected capacity change from 0 to 1099511627776 [ 9967.189174] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9967.194177] *** VALIDATE xfs *** [ 9967.200761] XFS (loop0): Mounting V5 Filesystem [ 9967.216259] XFS (loop0): Ending clean mount [ 9967.839484] loop1: detected capacity change from 0 to 18691697672192 [ 9995.910914] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10004.765820] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [10009.641134] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [10014.484867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10017.810649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10020.745679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10023.690814] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10026.732269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10026.772892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10026.893219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10026.911879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10026.955308] Lustre: lustre-MDT0000: new disk, initializing [10027.002170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10027.015325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10028.757949] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10034.475372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10034.508682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10034.552736] Lustre: 333935:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10034.586865] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10034.589424] Lustre: Skipped 1 previous similar message [10034.635569] Lustre: lustre-MDT0001: new disk, initializing [10034.716036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10034.721993] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10036.813923] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10041.433172] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10043.409811] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10046.904376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10046.959536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10047.100772] Lustre: lustre-OST0000: new disk, initializing [10047.107323] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10048.654965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10048.659553] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10048.716662] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10049.944803] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10054.098843] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10055.833106] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10055.921103] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10057.637323] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10057.729393] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10059.233443] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10059.233910] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10059.237337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10059.246735] Lustre: Skipped 1 previous similar message [10064.356243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10064.358989] Lustre: Skipped 2 previous similar messages [10073.055211] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10073.085238] LustreError: 336876:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [10073.090469] LustreError: 336876:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [10073.142959] Lustre: server umount lustre-OST0000 complete [10075.616211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10075.619139] LustreError: Skipped 1 previous similar message [10075.622067] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10075.628641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10075.631476] Lustre: Skipped 2 previous similar messages [10079.712944] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10080.999370] Lustre: server umount lustre-MDT0000 complete [10082.591460] LustreError: 334717:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353041 with bad export cookie 12439304932124471224 [10082.594874] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10082.596455] LustreError: 334717:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10082.741702] Lustre: server umount lustre-MDT0001 complete [10086.142483] XFS (loop0): Unmounting Filesystem [10088.115664] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 23:17:26 (1763353046) [10091.266463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10091.484971] LustreError: 338097:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10091.493088] LustreError: 338097:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10091.519724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10091.522774] Lustre: Skipped 2 previous similar messages [10093.226148] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10096.925181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10098.812941] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10101.203485] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10102.775905] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10106.016721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10108.925596] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10112.076114] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10122.720839] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10122.723779] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10122.730487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10122.733267] Lustre: Skipped 2 previous similar messages [10124.213989] LustreError: 340439:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [10124.220623] LustreError: 340439:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [10124.276791] Lustre: server umount lustre-OST0000 complete [10136.074971] Lustre: server umount lustre-MDT0000 complete [10137.056391] LustreError: 338097:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10137.071279] LustreError: 338097:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10137.982080] LustreError: 338077:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353097 with bad export cookie 12439304932124472106 [10137.982080] LustreError: 338076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763353097 with bad export cookie 12439304932124472106 [10137.982671] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10137.987264] LustreError: 338077:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [10138.168213] Lustre: server umount lustre-MDT0001 complete [10143.174657] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [10144.518864] Key type lgssc unregistered [10144.680186] LNet: 341375:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10144.686133] LNetError: 341375:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10144.698555] LNet: Removed LNI 192.168.201.158@tcp [10145.107107] Key type .llcrypt unregistered [10145.108657] Key type ._llcrypt unregistered [10158.159480] Key type ._llcrypt registered [10158.161029] Key type .llcrypt registered [10158.212928] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [10158.587346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10158.596219] alg: No test for adler32 (adler32-zlib) [10159.480633] Lustre: Lustre: Build Version: 2.16.61_47_g0e58e73 [10159.600815] LNet: Added LNI 192.168.201.158@tcp [8/256/0/180] [10161.207269] Key type lgssc registered [10161.687941] Lustre: Echo OBD driver; http://www.lustre.org/ [10165.216523] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10165.225723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10166.468484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10168.176322] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10172.097735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10172.284129] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10174.045416] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10176.416538] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10178.046583] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10181.332715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10181.534096] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10184.421883] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10187.145661] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10196.961580] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10196.961649] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10196.970393] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10196.973218] Lustre: Skipped 1 previous similar message [10198.699953] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [10202.081332] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10202.084539] Lustre: Skipped 1 previous similar message [10202.433771] LustreError: 345194:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10202.458288] Lustre: server umount lustre-MDT0001 complete [10203.572949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10203.811567] LustreError: 343611:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10203.818831] LustreError: 343611:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10206.678086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10206.738549] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10206.740820] Lustre: Found index 1 for lustre-MDT0001, updating log [10206.760719] Lustre: 342887:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10206.896467] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10208.651366] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10212.328143] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10212.337072] LustreError: 346216:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10212.339734] LustreError: 346216:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10212.343175] LustreError: 342891:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10212.366758] LustreError: 346224:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -17 [10212.371085] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10212.371085] [10514.407206] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10514.410943] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10514.416332] Lustre: Skipped 1 previous similar message [10514.419270] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10514.421212] Lustre: Skipped 1 previous similar message [10517.258119] LustreError: 356812:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10517.262346] LustreError: 356812:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10517.303971] Lustre: server umount lustre-MDT0001 complete [10518.569760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10519.521950] LustreError: 342899:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10519.533204] LustreError: 342899:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10521.252790] LustreError: 352707:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10522.149335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10522.212889] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10522.216915] Lustre: Found index 1 for lustre-MDT0001, updating log [10522.234303] Lustre: 342888:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10522.377824] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10524.223906] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10527.720590] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10527.732598] LustreError: 357796:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10527.736994] LustreError: 357796:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10527.742368] LustreError: 342891:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10527.746767] LustreError: 342891:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10527.783870] LustreError: 357803:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [10527.787248] LustreError: 357803:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -17 [10527.790571] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.158@tcp [10527.790571] [10828.470478] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.58@tcp (stopping) [10828.474043] Lustre: Skipped 1 previous similar message [10829.794834] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10829.795126] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10829.801912] Lustre: Skipped 1 previous similar message [10834.381475] LustreError: 368428:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10834.383664] LustreError: 368428:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10834.434144] Lustre: server umount lustre-MDT0001 complete [10834.914408] LustreError: 344034:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10834.925793] LustreError: 344034:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10835.992166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10839.728760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10839.791198] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10839.793231] Lustre: Found index 1 for lustre-MDT0001, updating log [10839.815440] Lustre: 342888:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10839.972515] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10841.998266] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [10845.156859] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10848.231276] LustreError: 369556:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10848.239446] LustreError: 369556:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10848.247820] LustreError: 342891:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10848.250884] LustreError: 342891:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10848.274572] LustreError: 369559:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10848.281749] LustreError: 369559:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.158@tcp: cfg command failed: rc = -114 [10848.286178] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10848.286178] [11162.593609] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11162.594125] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11162.596183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11162.596193] Lustre: Skipped 3 previous similar messages [11162.598573] Lustre: Skipped 3 previous similar messages [11172.319148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11172.361741] LustreError: 380411:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [11172.363725] LustreError: 380411:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11172.396472] Lustre: server umount lustre-MDT0000 complete [11172.834951] LustreError: 343611:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11172.842070] LustreError: 343611:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11173.902996] LustreError: 342881:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354133 with bad export cookie 14829502208685753484 [11173.906765] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11173.908015] LustreError: 342881:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [11177.953513] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11178.977685] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11178.980837] Lustre: Skipped 6 previous similar messages [11184.095909] LustreError: 344630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11184.102700] LustreError: 344630:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11188.191202] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11188.238075] LustreError: 380614:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [11188.241381] LustreError: 380614:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11188.308908] Lustre: server umount lustre-MDT0001 complete [11190.137600] Lustre: server umount lustre-OST0000 complete [11194.165666] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [11198.199052] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11202.380801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11205.086695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11207.541053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11210.253067] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11213.133872] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 23:36:11 (1763354171) [11215.910384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11215.942727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11216.047643] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11216.064231] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11216.110475] Lustre: lustre-MDT0000: new disk, initializing [11216.153668] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11216.164448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11217.839739] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11222.732155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11222.766525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [11222.815481] Lustre: 383347:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11222.848562] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11222.851458] Lustre: Skipped 1 previous similar message [11222.892842] Lustre: lustre-MDT0001: new disk, initializing [11222.941621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11222.961579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11222.967149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11224.734841] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11228.507553] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11229.942889] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11232.916902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11232.948279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [11233.084790] Lustre: lustre-OST0000: new disk, initializing [11233.088373] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [11233.091702] Lustre: Skipped 1 previous similar message [11233.093054] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11233.135488] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11235.632188] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_default_debug -1 all [11238.383223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11238.389248] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11238.412882] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11239.278934] Lustre: DEBUG MARKER: oleg158-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11243.489959] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11243.493639] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11243.499010] Lustre: Skipped 1 previous similar message [11248.579609] LustreError: 385793:0:(obd_class.h:479:obd_check_dev()) Device 20 not setup [11248.581817] LustreError: 385793:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [11248.609682] LustreError: 383358:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11248.619797] LustreError: 383358:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11248.629426] Lustre: server umount lustre-MDT0000 complete [11250.202878] LustreError: 384129:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763354209 with bad export cookie 14829502208685755423 [11250.206548] LustreError: 384141:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11250.209396] LustreError: MGC192.168.201.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11250.211509] LustreError: 384129:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [11250.349370] Lustre: server umount lustre-MDT0001 complete [11262.307468] Lustre: server umount lustre-OST0000 complete [11274.132710] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing set_hostid [11278.097401] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing load_modules_local [11282.390294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11285.302864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11288.036255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11290.977883] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11291.823883] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11076 sec ========= 23:37:30 (1763354250) [11292.550839] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 23:37:31 (1763354251) === [11294.019242] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 23:37:32 (1763354252) === [11310.524541] Lustre: DEBUG MARKER: oleg158-server.virtnet: executing unload_modules_local [11311.737275] Key type lgssc unregistered [11311.898750] LNet: 391300:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11311.902229] LNetError: 391300:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [11311.911483] LNet: Removed LNI 192.168.201.158@tcp [11312.281350] Key type .llcrypt unregistered [11312.282950] Key type ._llcrypt unregistered