[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 409229060 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002170] x2apic enabled [ 0.002984] Switched APIC routing to physical x2apic. [ 0.003948] kvm-guest: setup PV IPIs [ 0.005895] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006008] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007004] pid_max: default: 32768 minimum: 301 [ 0.008093] LSM: Security Framework initializing [ 0.009024] Yama: becoming mindful. [ 0.010018] SELinux: Initializing. [ 0.010727] *** VALIDATE selinux *** [ 0.017593] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.021120] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.022079] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023061] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024052] *** VALIDATE tmpfs *** [ 0.025325] *** VALIDATE proc *** [ 0.026161] *** VALIDATE cgroup *** [ 0.027004] *** VALIDATE cgroup2 *** [ 0.028174] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.029098] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.030003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.031016] Spectre V2 : User space: Vulnerable [ 0.032003] Speculative Store Bypass: Vulnerable [ 0.034940] debug: unmapping init [mem 0xffffffffb7259000-0xffffffffb7260fff] [ 0.036138] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.037422] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.038010] ... version: 2 [ 0.038777] ... bit width: 48 [ 0.039006] ... generic registers: 4 [ 0.039755] ... value mask: 0000ffffffffffff [ 0.040009] ... max period: 00007fffffffffff [ 0.041006] ... fixed-purpose events: 3 [ 0.041761] ... event mask: 000000070000000f [ 0.042125] rcu: Hierarchical SRCU implementation. [ 0.044088] smp: Bringing up secondary CPUs ... [ 0.045338] x86: Booting SMP configuration: [ 0.046014] .... node #0, CPUs: #1 #2 #3 [ 0.048301] smp: Brought up 1 node, 4 CPUs [ 0.049782] smpboot: Max logical packages: 1 [ 0.050007] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.142787] node 0 deferred pages initialised in 92ms [ 0.145238] devtmpfs: initialized [ 0.146177] x86/mm: Memory block size: 128MB [ 0.148328] gcov: version magic: 0x41383552 [ 0.150174] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.152055] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.153188] pinctrl core: initialized pinctrl subsystem [ 0.154074] [ 0.154449] ************************************************************* [ 0.156006] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157005] ** ** [ 0.159006] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.160005] ** ** [ 0.161005] ** This means that this kernel is built to expose internal ** [ 0.162005] ** IOMMU data structures, which may compromise security on ** [ 0.163007] ** your system. ** [ 0.165005] ** ** [ 0.166005] ** If you see this message and you are not debugging the ** [ 0.167005] ** kernel, report this immediately to your vendor! ** [ 0.168005] ** ** [ 0.169005] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.171006] ************************************************************* [ 0.172414] NET: Registered protocol family 16 [ 0.173301] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.175026] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.176025] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.178236] cpuidle: using governor menu [ 0.179334] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.180253] PCI: Using configuration type 1 for base access [ 0.182103] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.188037] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.189027] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.191099] cryptd: max_cpu_qlen set to 1000 [ 0.193137] ACPI: Added _OSI(Module Device) [ 0.194006] ACPI: Added _OSI(Processor Device) [ 0.194984] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.196009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.199079] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.202416] ACPI: Interpreter enabled [ 0.203045] ACPI: PM: (supports S0 S3 S4 S5) [ 0.203967] ACPI: Using IOAPIC for interrupt routing [ 0.205062] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.207342] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.216334] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.219019] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.222011] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.225041] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.230285] acpiphp: Slot [2] registered [ 0.232070] acpiphp: Slot [5] registered [ 0.233084] acpiphp: Slot [6] registered [ 0.235068] acpiphp: Slot [7] registered [ 0.236065] acpiphp: Slot [8] registered [ 0.237068] acpiphp: Slot [9] registered [ 0.239066] acpiphp: Slot [10] registered [ 0.240069] acpiphp: Slot [3] registered [ 0.242067] acpiphp: Slot [4] registered [ 0.243074] acpiphp: Slot [11] registered [ 0.245058] acpiphp: Slot [12] registered [ 0.246060] acpiphp: Slot [13] registered [ 0.247097] acpiphp: Slot [14] registered [ 0.249062] acpiphp: Slot [15] registered [ 0.250076] acpiphp: Slot [16] registered [ 0.252057] acpiphp: Slot [17] registered [ 0.253057] acpiphp: Slot [18] registered [ 0.255059] acpiphp: Slot [19] registered [ 0.256071] acpiphp: Slot [20] registered [ 0.258082] acpiphp: Slot [21] registered [ 0.259053] acpiphp: Slot [22] registered [ 0.261054] acpiphp: Slot [23] registered [ 0.262049] acpiphp: Slot [24] registered [ 0.263050] acpiphp: Slot [25] registered [ 0.264051] acpiphp: Slot [26] registered [ 0.266061] acpiphp: Slot [27] registered [ 0.267050] acpiphp: Slot [28] registered [ 0.268049] acpiphp: Slot [29] registered [ 0.269050] acpiphp: Slot [30] registered [ 0.271050] acpiphp: Slot [31] registered [ 0.272064] PCI host bridge to bus 0000:00 [ 0.273011] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.276012] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.278011] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.280009] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.282011] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.285010] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.287104] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.289643] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.291837] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.300007] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.303035] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.305010] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.308010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.310008] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.312344] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.314565] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.317053] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.320243] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.324008] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.336009] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.341009] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.346730] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.351008] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.357008] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.366008] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.374496] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.378009] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.386009] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.398008] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.404189] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.408008] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.412008] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.421008] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.427095] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.430968] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.434011] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.442965] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.449143] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.454007] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.458009] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.467010] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.474618] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.478997] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.482010] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.491012] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.498705] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.500240] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.502232] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.504217] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.505106] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.509020] iommu: Default domain type: Passthrough [ 0.509918] SCSI subsystem initialized [ 0.510034] ACPI: bus type USB registered [ 0.510815] usbcore: registered new interface driver usbfs [ 0.512035] usbcore: registered new interface driver hub [ 0.513027] usbcore: registered new device driver usb [ 0.514072] pps_core: LinuxPPS API ver. 1 registered [ 0.515006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.517029] PTP clock support registered [ 0.518073] EDAC MC: Ver: 3.0.0 [ 0.519125] PCI: Using ACPI for IRQ routing [ 0.520576] NetLabel: Initializing [ 0.521007] NetLabel: domain hash size = 128 [ 0.521871] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.523037] NetLabel: unlabeled traffic allowed by default [ 0.524072] vgaarb: loaded [ 0.525182] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.525994] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.532216] clocksource: Switched to clocksource kvm-clock [ 0.610389] VFS: Disk quotas dquot_6.6.0 [ 0.611287] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.612771] *** VALIDATE ramfs *** [ 0.613431] *** VALIDATE hugetlbfs *** [ 0.614263] pnp: PnP ACPI init [ 0.615786] pnp: PnP ACPI: found 6 devices [ 0.627943] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.630979] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.632985] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.635018] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.637220] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.639619] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.642112] NET: Registered protocol family 2 [ 0.644191] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.648360] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.650337] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.653892] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.655751] TCP: Hash tables configured (established 65536 bind 65536) [ 0.657325] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.659064] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.660579] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.662116] NET: Registered protocol family 1 [ 0.663990] RPC: Registered named UNIX socket transport module. [ 0.665087] RPC: Registered udp transport module. [ 0.665984] RPC: Registered tcp transport module. [ 0.666860] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.668119] NET: Registered protocol family 44 [ 0.668982] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.670129] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.671241] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.672655] PCI: CLS 0 bytes, default 64 [ 0.673536] Unpacking initramfs... [ 1.833260] debug: unmapping init [mem 0xffff904c7cc54000-0xffff904c7ffbffff] [ 1.836613] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.838721] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 1.841421] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.282865] Initialise system trusted keyrings [ 2.284022] Key type blacklist registered [ 2.285185] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.291669] zbud: loaded [ 2.293623] *** VALIDATE nfs *** [ 2.294313] *** VALIDATE nfs4 *** [ 2.295233] pstore: using deflate compression [ 2.297089] Platform Keyring initialized [ 2.360311] NET: Registered protocol family 38 [ 2.362176] Key type asymmetric registered [ 2.363592] Asymmetric key parser 'x509' registered [ 2.364661] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.366275] io scheduler mq-deadline registered [ 2.367955] io scheduler kyber registered [ 2.369542] io scheduler bfq registered [ 2.371228] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.373950] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.376553] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.379181] ACPI: Power Button [PWRF] [ 2.383503] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.388430] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.396226] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.400688] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.409820] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.435563] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.461828] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.465386] Non-volatile memory driver v1.3 [ 2.466880] Linux agpgart interface v0.103 [ 2.488667] virtio_blk virtio1: [vda] 134224 512-byte logical blocks (68.7 MB/65.5 MiB) [ 2.491657] vda: detected capacity change from 0 to 68722688 [ 2.502552] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.505516] vdb: detected capacity change from 0 to 1073741824 [ 2.517443] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.520052] vdc: detected capacity change from 0 to 2621440000 [ 2.533268] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.535907] vdd: detected capacity change from 0 to 2621440000 [ 2.549210] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.551990] vde: detected capacity change from 0 to 4294967296 [ 2.563772] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.565462] vdf: detected capacity change from 0 to 4294967296 [ 2.569102] libphy: Fixed MDIO Bus: probed [ 2.572318] usbcore: registered new interface driver usbserial_generic [ 2.573744] usbserial: USB Serial support registered for generic [ 2.575264] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.578072] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.579195] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.580609] mousedev: PS/2 mouse device common for all mice [ 2.582992] rtc_cmos 00:05: RTC can wake from S4 [ 2.585124] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.586838] rtc_cmos 00:05: registered as rtc0 [ 2.588418] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.588910] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.590199] intel_pstate: CPU model not supported [ 2.592752] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.595540] hid: raw HID events driver (C) Jiri Kosina [ 2.596772] usbcore: registered new interface driver usbhid [ 2.598288] usbhid: USB HID core driver [ 2.599148] drop_monitor: Initializing network drop monitor service [ 2.600461] Initializing XFRM netlink socket [ 2.601607] NET: Registered protocol family 10 [ 2.603484] Segment Routing with IPv6 [ 2.604410] NET: Registered protocol family 17 [ 2.605591] mpls_gso: MPLS GSO support [ 2.609477] RAS: Correctable Errors collector initialized. [ 2.610722] AVX version of gcm_enc/dec engaged. [ 2.611727] AES CTR mode by8 optimization enabled [ 2.666378] sched_clock: Marking stable (2666361607, 0)->(3260210600, -593848993) [ 2.668777] registered taskstats version 1 [ 2.669952] Loading compiled-in X.509 certificates [ 2.671364] zswap: loaded using pool lzo/zbud [ 2.687092] Key type big_key registered [ 2.697820] Key type encrypted registered [ 2.698741] ima: No TPM chip found, activating TPM-bypass! [ 2.699794] ima: Allocated hash algorithm: sha1 [ 2.700701] ima: No architecture policies found [ 2.701639] evm: Initialising EVM extended attributes: [ 2.702778] evm: security.selinux [ 2.703731] evm: security.ima [ 2.704513] evm: security.capability [ 2.705229] evm: HMAC attrs: 0x1 [ 2.706620] rtc_cmos 00:05: setting system clock to 2026-02-07 06:51:24 UTC (1770447084) [ 2.710403] debug: unmapping init [mem 0xffffffffb8203000-0xffffffffb83fffff] [ 2.712091] debug: unmapping init [mem 0xffffffffb6f82000-0xffffffffb7258fff] [ 2.719094] Write protecting the kernel read-only data: 28672k [ 2.722340] debug: unmapping init [mem 0xffffffffb5603000-0xffffffffb57fffff] [ 2.723940] debug: unmapping init [mem 0xffffffffb5f14000-0xffffffffb5ffffff] [ 2.746024] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 2.751113] systemd[1]: Detected virtualization kvm. [ 2.752110] systemd[1]: Detected architecture x86-64. [ 2.753151] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 2.771775] systemd[1]: No hostname configured. [ 2.772759] systemd[1]: Set hostname to . [ 2.773908] random: systemd: uninitialized urandom read (16 bytes read) [ 2.775356] systemd[1]: Initializing machine ID from random generator. [ 2.806318] random: ln: uninitialized urandom read (6 bytes read) [ 2.868133] random: systemd: uninitialized urandom read (16 bytes read) [ 2.869532] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 2.872134] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 2.874298] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.297020] device-mapper: uevent: version 1.0.3 [ 3.298409] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 3.761020] virtio_net virtio0 ens2: renamed from eth0 [ 3.774071] random: fast init done [ 3.826587] scsi host0: ata_piix [ 3.883276] scsi host1: ata_piix [ 3.884487] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 3.885818] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 7.883067] dracut-initqueue[592]: RTNETLINK answers: File exists [ 8.208515] random: crng init done [ 8.209479] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 8.511712] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 9.358404] printk: systemd: 26 output lines suppressed due to ratelimiting [ 9.516474] SELinux: Disabled at runtime. [ 9.567582] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 9.572141] systemd[1]: Detected virtualization kvm. [ 9.573248] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 9.892254] systemd[1]: initrd-switch-root.service: Succeeded. [ 9.894324] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 9.897021] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 9.899091] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 9.901115] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 9.905046] systemd[1]: Starting Journal Service... Starting Journal Service... [ 9.923746] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Mounting Huge Pages File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-getty.slice. [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 10.034292] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 10.260885] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 10.497204] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 10.538350] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 10.623763] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 10.630141] EDAC sbridge: Ver: 1.1.2 [ 11.825349] Key type dns_resolver registered [ 12.097429] NFS: Registering the id_resolver key type [ 12.098585] Key type id_resolver registered [ 12.099450] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg225-server login: [ 34.169590] libcfs: loading out-of-tree module taints kernel. [ 34.186120] Key type ._llcrypt registered [ 34.186963] Key type .llcrypt registered [ 34.222310] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 41.669492] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 42.245486] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 42.251259] alg: No test for adler32 (adler32-zlib) [ 43.241679] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 43.515488] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 45.127198] Key type lgssc registered [ 45.656876] Lustre: Echo OBD driver; http://www.lustre.org/ [ 51.449804] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 64.267291] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 68.464777] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 68.473957] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 69.568160] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 69.581960] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 69.622865] Lustre: lustre-MDT0000: new disk, initializing [ 69.649224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 69.656240] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 70.856893] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 75.879652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 75.916762] Lustre: 6476:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 75.929684] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 75.932295] Lustre: Skipped 1 previous similar message [ 75.978165] Lustre: lustre-MDT0001: new disk, initializing [ 75.998847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 76.006711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 76.009573] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 77.202857] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 79.412519] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 82.966866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 83.059957] Lustre: lustre-OST0000: new disk, initializing [ 83.061812] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 83.076331] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 84.954300] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 86.444896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 86.447872] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 86.470207] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 91.202496] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 91.247205] Lustre: lustre-OST0001: new disk, initializing [ 91.248870] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 91.264097] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 93.420452] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 95.731664] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 95.739735] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 95.774065] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 99.472250] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 105.227673] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 112.166498] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing check_logdir /tmp/testlogs/ [ 113.335477] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing yml_node [ 114.585781] Lustre: DEBUG MARKER: Client: 2.17.50.44 [ 115.493863] Lustre: DEBUG MARKER: MDS: 2.17.50.44 [ 116.679020] Lustre: DEBUG MARKER: OSS: 2.17.50.44 [ 117.472492] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Feb 7 01:53:18 EST 2026 [ 126.205742] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 126.908681] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 141.792570] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 141.793299] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 141.801977] Lustre: Skipped 3 previous similar messages [ 141.808817] Lustre: Skipped 3 previous similar messages [ 145.610040] LustreError: 12169:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 145.669690] Lustre: server umount lustre-MDT0000 complete [ 147.448100] LustreError: 8382:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447229 with bad export cookie 16629442384191342674 [ 147.449152] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 147.452940] LustreError: 8382:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 1 previous similar message [ 152.032737] LustreError: 6484:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 152.033345] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 152.034326] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 152.039570] LustreError: 6484:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 6 previous similar messages [ 152.045859] Lustre: Skipped 1 previous similar message [ 153.607421] LustreError: 12371:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 153.612285] LustreError: 12371:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 153.704286] Lustre: server umount lustre-MDT0001 complete [ 155.706183] LustreError: 12572:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 155.709503] LustreError: 12572:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 155.740693] Lustre: server umount lustre-OST0000 complete [ 157.691516] Lustre: server umount lustre-OST0001 complete [ 160.850604] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 172.060774] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 193.173239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.369331] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 193.412200] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 193.478736] Lustre: lustre-MDT0000: new disk, initializing [ 193.545249] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.566589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 195.787957] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 201.775673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 201.852685] Lustre: 14941:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 201.904300] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 201.907053] Lustre: Skipped 1 previous similar message [ 201.994278] Lustre: lustre-MDT0001: new disk, initializing [ 202.064942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 202.091527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 202.099298] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 204.121891] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 208.561579] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 210.192137] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 213.798190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 215.845233] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 215.917103] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 217.116441] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 221.797166] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 224.132490] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 224.259491] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 226.819694] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 226.956388] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 231.396854] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 231.398843] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 231.404367] LustreError: Skipped 1 previous similar message [ 231.405622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 231.415660] Lustre: Skipped 1 previous similar message [ 231.419911] Lustre: Skipped 1 previous similar message [ 234.174582] LustreError: 17870:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 234.181316] LustreError: 17870:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 234.308294] Lustre: server umount lustre-OST0000 complete [ 238.048356] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 238.052680] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 238.061922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 242.762900] LustreError: 18071:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 242.769429] LustreError: 18071:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 242.813996] Lustre: server umount lustre-MDT0000 complete [ 244.634376] LustreError: 14934:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447326 with bad export cookie 16629442384191344193 [ 244.634929] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 244.639422] LustreError: 14934:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 1 previous similar message [ 244.787184] Lustre: server umount lustre-MDT0001 complete [ 247.004162] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:55:27 (1770447327) [ 250.200917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 250.401469] LustreError: 18796:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 250.443178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 250.447691] Lustre: Skipped 1 previous similar message [ 252.277214] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 255.463483] LustreError: 18796:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 256.110968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 258.214256] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 260.738537] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 262.273473] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 265.250015] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 267.676274] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 270.005480] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 282.744527] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 287.200614] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 287.203961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 287.209189] Lustre: Skipped 1 previous similar message [ 287.211612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 287.213931] Lustre: Skipped 2 previous similar messages [ 292.610837] LustreError: 21114:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 292.613979] LustreError: 21114:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 292.662960] Lustre: server umount lustre-MDT0000 complete [ 294.486707] LustreError: 18777:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447376 with bad export cookie 16629442384191345068 [ 294.495692] LustreError: 18777:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 294.495807] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 294.670438] Lustre: server umount lustre-MDT0001 complete [ 307.186257] Lustre: server umount lustre-OST0000 complete [ 319.529159] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 325.279740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 325.505198] LustreError: 22842:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 325.535924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 325.538616] Lustre: Skipped 2 previous similar messages [ 327.756788] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 330.720411] LustreError: 22842:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 333.217703] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 335.655655] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 337.391884] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 341.434770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 345.124533] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 350.330536] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 350.435962] Lustre: lustre-OST0001: new disk, initializing [ 350.441271] Lustre: Skipped 1 previous similar message [ 350.444312] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 350.446547] Lustre: Skipped 1 previous similar message [ 351.897837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 351.902318] Lustre: Skipped 1 previous similar message [ 351.904423] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 351.954808] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 353.801577] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 360.546840] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 363.024489] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 376.906652] LustreError: 26320:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 376.910145] LustreError: 26320:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 376.952659] Lustre: server umount lustre-MDT0000 complete [ 377.829567] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 377.831541] LustreError: 22837:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 377.836861] Lustre: Skipped 5 previous similar messages [ 377.846144] LustreError: 22837:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 2 previous similar messages [ 378.992257] LustreError: 25110:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447460 with bad export cookie 16629442384191345922 [ 378.995875] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 378.998214] LustreError: 25110:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 379.172596] Lustre: server umount lustre-MDT0001 complete [ 392.215581] Lustre: server umount lustre-OST0000 complete [ 403.398847] Lustre: server umount lustre-OST0001 complete [ 410.662940] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 414.715847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 414.955892] LustreError: 28095:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 415.025394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 415.029185] Lustre: Skipped 3 previous similar messages [ 416.626550] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 421.315751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 424.367435] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 426.016059] Lustre: 29207:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 429.434633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 433.439419] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 438.557422] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 441.624837] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 445.840740] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 447.538560] Lustre: 31059:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 455.138380] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 455.148408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 455.151874] Lustre: Skipped 3 previous similar messages [ 455.163428] Lustre: Skipped 6 previous similar messages [ 460.872778] LustreError: 31261:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 460.875862] LustreError: 31261:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 460.919764] Lustre: server umount lustre-MDT0000 complete [ 462.524577] LustreError: 31061:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447544 with bad export cookie 16629442384191347217 [ 462.527534] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 462.529986] LustreError: 31061:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 462.670266] Lustre: server umount lustre-MDT0001 complete [ 474.661084] Lustre: server umount lustre-OST0000 complete [ 486.449293] Lustre: server umount lustre-OST0001 complete [ 490.411008] hrtimer: interrupt took 2819147 ns [ 492.047431] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:59:32 (1770447572) [ 509.052767] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 514.505253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 514.750657] LustreError: 33841:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 514.765626] LustreError: 33841:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 4 previous similar messages [ 517.333182] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 523.030265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 525.856511] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 527.877782] Lustre: 34956:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 532.151320] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 536.396326] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 542.000613] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 545.840282] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 551.081547] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 553.267777] Lustre: 36804:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 562.661189] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 562.672294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 562.678686] Lustre: Skipped 7 previous similar messages [ 567.117908] LustreError: 36956:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 567.120581] LustreError: 36956:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 567.180304] Lustre: server umount lustre-MDT0000 complete [ 569.451305] LustreError: 33822:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447651 with bad export cookie 16629442384191348323 [ 569.452977] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 569.465430] LustreError: 33822:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 1 previous similar message [ 569.648986] Lustre: server umount lustre-MDT0001 complete [ 582.168869] Lustre: server umount lustre-OST0000 complete [ 594.591709] Lustre: server umount lustre-OST0001 complete [ 599.493800] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 02:01:20 (1770447680) [ 608.959962] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 615.630258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 615.904967] LustreError: 38942:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 615.920209] LustreError: 38942:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 9 previous similar messages [ 615.996078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 615.999775] Lustre: Skipped 7 previous similar messages [ 618.217931] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 623.119398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 625.561366] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 627.124524] Lustre: 40055:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 630.846453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 634.136304] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 639.040940] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 642.025441] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 646.314828] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 653.462250] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 653.465943] Lustre: Skipped 1 previous similar message [ 655.845435] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 655.847036] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 655.853059] Lustre: Skipped 4 previous similar messages [ 655.858522] Lustre: Skipped 3 previous similar messages [ 661.208899] Lustre: server umount lustre-MDT0000 complete [ 663.320264] LustreError: 38922:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447745 with bad export cookie 16629442384191349429 [ 663.322034] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 663.328685] LustreError: 38922:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 669.606905] Lustre: server umount lustre-MDT0001 complete [ 674.528872] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 688.608257] LustreError: 3627:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff904bc2191f80 x1856448364874752/t0(0) o250->MGC192.168.202.125@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 690.895806] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 695.352851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 697.596205] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 698.666207] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 700.440516] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 701.302246] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 703.148363] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 704.113850] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 712.323944] LustreError: 44299:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 712.326102] LustreError: 44299:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 712.360738] Lustre: server umount lustre-MDT0000 complete [ 714.430910] LustreError: 41961:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447796 with bad export cookie 16629442384191350619 [ 714.438666] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 714.440591] LustreError: 41961:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 714.657721] Lustre: server umount lustre-MDT0001 complete [ 726.560732] Lustre: server umount lustre-OST0000 complete [ 738.814406] Lustre: server umount lustre-OST0001 complete [ 742.874890] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 02:03:43 (1770447823) [ 750.738968] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 756.030996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 756.257087] LustreError: 46268:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 756.271807] LustreError: 46268:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 17 previous similar messages [ 758.763374] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 763.373627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 765.583377] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 767.250461] Lustre: 47385:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 767.256503] Lustre: 47385:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 770.709488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 773.778499] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 778.488371] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 781.734238] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 786.298214] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 793.595907] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 799.712650] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 799.716652] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 799.721859] Lustre: Skipped 8 previous similar messages [ 799.724517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 799.727215] Lustre: Skipped 15 previous similar messages [ 805.230041] Lustre: server umount lustre-MDT0000 complete [ 806.990047] LustreError: 46248:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447888 with bad export cookie 16629442384191351403 [ 806.990047] LustreError: 47387:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770447888 with bad export cookie 16629442384191351403 [ 806.990939] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 806.996897] LustreError: 47387:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 807.125231] Lustre: server umount lustre-MDT0001 complete [ 818.265744] Lustre: server umount lustre-OST0000 complete [ 830.577954] Lustre: server umount lustre-OST0001 complete [ 835.648729] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 02:05:16 (1770447916) [ 837.061022] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 837.943665] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 02:05:18 (1770447918) [ 843.761888] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 848.046919] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 859.405909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 859.577557] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 859.625111] Lustre: lustre-MDT0000: new disk, initializing [ 859.672862] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 861.722309] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 866.122179] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 869.852114] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 872.001266] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 872.026643] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 873.533884] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 878.054033] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 879.210287] Lustre: DEBUG MARKER: create test files [ 896.835743] Lustre: server umount lustre-OST0000 complete [ 908.361870] Lustre: server umount lustre-MDT0000 complete [ 913.273067] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 914.699928] Key type lgssc unregistered [ 914.870299] LNet: 54776:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 914.875372] LNetError: 54776:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 914.890478] LNet: Removed LNI 192.168.202.125@tcp [ 915.344189] Key type .llcrypt unregistered [ 915.346640] Key type ._llcrypt unregistered [ 929.716922] Key type ._llcrypt registered [ 929.719166] Key type .llcrypt registered [ 929.791509] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 930.274927] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 930.332760] alg: No test for adler32 (adler32-zlib) [ 931.265014] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 931.390077] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 932.991155] Key type lgssc registered [ 933.543585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 937.617541] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 937.643145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 938.949884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 940.981271] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 943.582181] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 946.954582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 947.159918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 949.227336] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 949.281957] Lustre: 57354:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 949.289856] Lustre: 57354:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 949.293959] Lustre: 57354:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 949.299257] Lustre: 57354:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 949.304476] Lustre: 57354:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 949.308672] Lustre: 57354:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 950.280299] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 952.805683] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 957.316752] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 959.898173] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 964.579626] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 964.583623] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 964.593189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 968.434321] LustreError: 58098:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 968.496090] Lustre: server umount lustre-OST0000 complete [ 970.415640] LustreError: 58303:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 970.420351] LustreError: 58303:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 970.536761] Lustre: server umount lustre-MDT0000 complete [ 975.518484] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 976.847574] Key type lgssc unregistered [ 976.994617] LNet: 58837:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 977.000235] LNetError: 58837:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 977.009863] LNet: Removed LNI 192.168.202.125@tcp [ 977.393886] Key type .llcrypt unregistered [ 977.396135] Key type ._llcrypt unregistered [ 991.914742] Key type ._llcrypt registered [ 991.917848] Key type .llcrypt registered [ 991.972890] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 992.413325] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 992.466402] alg: No test for adler32 (adler32-zlib) [ 993.351847] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 993.466689] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 995.063192] Key type lgssc registered [ 995.506379] Lustre: Echo OBD driver; http://www.lustre.org/ [ 998.536250] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 998.547047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 999.712676] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1001.241382] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1003.463084] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1006.416218] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1006.570481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1008.944098] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1009.643735] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1009.682867] Lustre: 61518:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1009.685946] Lustre: 61518:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1009.688584] Lustre: 61518:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1009.691022] Lustre: 61518:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1009.695906] Lustre: 61518:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1009.700470] Lustre: 61518:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1011.244900] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1014.590937] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1019.503086] LustreError: 62208:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1019.541102] Lustre: server umount lustre-OST0000 complete [ 1031.139149] LustreError: 62409:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1031.142241] LustreError: 62409:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1031.201514] Lustre: server umount lustre-MDT0000 complete [ 1035.827389] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 1037.055571] Key type lgssc unregistered [ 1037.206540] LNet: 62942:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1037.209841] LNetError: 62942:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1037.217360] LNet: Removed LNI 192.168.202.125@tcp [ 1037.579783] Key type .llcrypt unregistered [ 1037.581205] Key type ._llcrypt unregistered [ 1044.583889] Key type ._llcrypt registered [ 1044.585188] Key type .llcrypt registered [ 1044.636929] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 1051.731344] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1052.209124] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1052.220615] alg: No test for adler32 (adler32-zlib) [ 1053.108881] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 1053.217104] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 1054.807510] Key type lgssc registered [ 1055.272287] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1069.304081] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1069.312301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1070.426642] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1070.441429] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1070.481499] Lustre: lustre-MDT0000: new disk, initializing [ 1070.511338] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1070.519751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1072.075273] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1076.756105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1076.803764] Lustre: 66031:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1076.832376] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1076.835228] Lustre: Skipped 1 previous similar message [ 1076.876664] Lustre: lustre-MDT0001: new disk, initializing [ 1076.916821] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1076.932096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1076.936380] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1078.395334] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1081.896218] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1083.407407] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1085.988255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1086.105254] Lustre: lustre-OST0000: new disk, initializing [ 1086.107740] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1086.155273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1086.203414] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1086.209661] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1086.269314] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1088.442878] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1091.929379] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1093.418175] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1093.491909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1094.941258] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1095.025700] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1096.674850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1096.675601] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1096.676321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1096.680088] Lustre: Skipped 1 previous similar message [ 1101.794564] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1101.798530] Lustre: Skipped 2 previous similar messages [ 1106.914507] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1106.919911] Lustre: Skipped 1 previous similar message [ 1109.983127] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1110.001110] LustreError: 68959:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1110.043111] Lustre: server umount lustre-OST0000 complete [ 1112.032765] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1112.033490] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1112.037907] Lustre: Skipped 1 previous similar message [ 1117.152321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1117.155319] Lustre: Skipped 2 previous similar messages [ 1117.635470] LustreError: 69161:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1117.638288] LustreError: 69161:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1117.666862] Lustre: server umount lustre-MDT0000 complete [ 1119.036442] LustreError: 66024:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448200 with bad export cookie 4457937863338317385 [ 1119.038954] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1119.041289] LustreError: 66024:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1119.116427] LustreError: 69362:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1119.119103] LustreError: 69362:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1119.177242] Lustre: server umount lustre-MDT0001 complete [ 1122.514983] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:10:03 (1770448203) [ 1127.014815] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1141.914759] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 1145.558739] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1159.998762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1160.100222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1160.113935] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1160.153307] Lustre: lustre-MDT0000: new disk, initializing [ 1160.185117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1160.193030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1161.770659] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1166.479980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1166.523063] Lustre: 73578:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1166.547516] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1166.550822] Lustre: Skipped 1 previous similar message [ 1166.594281] Lustre: lustre-MDT0001: new disk, initializing [ 1166.637490] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1166.652051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1166.656499] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1168.181315] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1171.555530] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1172.865127] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1175.581132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1175.713767] Lustre: lustre-OST0000: new disk, initializing [ 1175.716692] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1175.752369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1177.870674] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1178.813149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1178.817303] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1178.854451] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1181.272590] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1182.873415] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1182.959889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1184.408288] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1184.486683] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1189.344837] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1189.345297] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1189.347873] LustreError: Skipped 2 previous similar messages [ 1189.353224] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1189.356447] Lustre: Skipped 1 previous similar message [ 1191.343095] LustreError: 76506:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1191.346275] LustreError: 76506:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1191.390994] Lustre: server umount lustre-OST0000 complete [ 1194.464493] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1194.470252] Lustre: Skipped 2 previous similar messages [ 1207.263096] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1207.300398] LustreError: 76710:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1207.303275] LustreError: 76710:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1207.333659] Lustre: server umount lustre-MDT0000 complete [ 1208.748804] LustreError: 74353:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448290 with bad export cookie 4457937863338318253 [ 1208.752215] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1208.753262] LustreError: 74353:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1208.870622] Lustre: server umount lustre-MDT0001 complete [ 1212.122520] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:11:33 (1770448293) [ 1214.787889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1214.980464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1216.600522] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1219.937325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1221.506187] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1223.467805] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1224.835969] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1227.443660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1229.672483] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1232.037587] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1233.473963] Lustre: 79909:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 444a2f21-55c3-4396-865b-5002a048f66c at adminstrative request [ 1234.914817] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1234.919713] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1245.155499] Lustre: MGS: Client 444a2f21-55c3-4396-865b-5002a048f66c (at 0@lo) reconnecting [ 1245.161331] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x3dddc6de170b5d11 to 0x3dddc6de170b5f80 [ 1245.166277] Lustre: MGC192.168.202.125@tcp: Connection restored to 0@lo (at 0@lo) [ 1266.332176] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1266.413829] Lustre: lustre-OST0001: new disk, initializing [ 1266.417702] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1266.457610] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1266.461126] Lustre: Skipped 2 previous similar messages [ 1268.533099] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1271.851481] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1272.332888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1272.337494] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1272.375792] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1277.413197] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1277.413812] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1277.422221] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1277.424877] Lustre: Skipped 7 previous similar messages [ 1279.020633] LustreError: 81046:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1279.022639] LustreError: 81046:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1279.043946] Lustre: server umount lustre-OST0001 complete [ 1280.992418] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1280.996170] LustreError: Skipped 2 previous similar messages [ 1282.531343] LustreError: 80967:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1282.537401] LustreError: 80967:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 1286.537530] Lustre: server umount lustre-OST0000 complete [ 1291.745236] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1291.748283] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1291.753580] Lustre: Skipped 3 previous similar messages [ 1293.917988] Lustre: server umount lustre-MDT0000 complete [ 1295.260235] LustreError: 77561:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448377 with bad export cookie 4457937863338319744 [ 1295.263341] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1295.264586] LustreError: 77561:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1295.333904] LustreError: 81651:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 1295.336492] LustreError: 81651:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 1295.393736] Lustre: server umount lustre-MDT0001 complete [ 1298.524726] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:12:59 (1770448379) [ 1299.091541] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1299.758484] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:13:00 (1770448380) [ 1300.350818] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1301.079972] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:13:02 (1770448382) [ 1301.629454] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1302.265050] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:13:03 (1770448383) [ 1302.838271] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1303.479292] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:13:04 (1770448384) [ 1304.085146] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1306.370622] loop: module loaded [ 1306.377455] loop0: detected capacity change from 0 to 4294967296 [ 1306.391859] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1306.991519] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1308.132875] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1309.700824] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1319.323209] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1319.338654] LustreError: 84292:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1319.341367] LustreError: 84292:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1319.343865] LustreError: 84292:0:(tgt_mount.c:2451:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1319.346429] LustreError: 84292:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1325.460482] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 1328.599090] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1341.482271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1341.578875] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1341.591420] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1341.629249] Lustre: lustre-MDT0000: new disk, initializing [ 1341.657324] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1341.666565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1343.148960] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1347.507884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1347.547985] Lustre: 87117:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1347.649191] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1349.023506] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1352.285542] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1353.644188] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1356.080755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1358.118863] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1358.167120] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1361.402959] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1362.869456] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1362.939871] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1364.273159] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1364.344620] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1368.545534] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1368.545628] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1368.547594] LustreError: Skipped 1 previous similar message [ 1368.551201] Lustre: Skipped 1 previous similar message [ 1368.552874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1368.555655] Lustre: Skipped 8 previous similar messages [ 1379.295099] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1379.313072] LustreError: 90048:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1379.314921] LustreError: 90048:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 1379.350243] Lustre: server umount lustre-OST0000 complete [ 1386.841509] Lustre: server umount lustre-MDT0000 complete [ 1388.079175] LustreError: 87894:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448469 with bad export cookie 4457937863338320353 [ 1388.082804] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1388.084428] LustreError: 87894:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1388.201195] Lustre: server umount lustre-MDT0001 complete [ 1391.135391] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:14:32 (1770448472) [ 1393.375976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1393.523415] LustreError: 91121:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1394.741242] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1397.695376] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1399.238902] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1401.155029] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1402.486070] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1404.833023] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1406.807978] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1408.534549] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1409.806807] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1419.049372] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec [ 1420.389165] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1420.463119] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1422.838178] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1422.897200] Lustre: lustre-OST0001: new disk, initializing [ 1422.898760] Lustre: Skipped 2 previous similar messages [ 1422.900589] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1422.903173] Lustre: Skipped 3 previous similar messages [ 1424.960842] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1428.072233] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1429.520576] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1430.020737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1430.025134] Lustre: Skipped 2 previous similar messages [ 1430.027430] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1430.031376] Lustre: Skipped 1 previous similar message [ 1430.062386] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1430.619560] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1431.976942] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1432.057333] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1441.877391] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1447.142634] Lustre: Failing over lustre-MDT0000 [ 1447.457134] LustreError: 96066:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1447.459225] LustreError: 96066:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1447.487368] Lustre: server umount lustre-MDT0000 complete [ 1448.892619] LustreError: 94188:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1448.897860] LustreError: 94188:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 1448.928408] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1448.931777] LustreError: Skipped 1 previous similar message [ 1448.933226] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1448.937215] Lustre: Skipped 2 previous similar messages [ 1454.002470] LustreError: 91115:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1454.011475] LustreError: 91115:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 6 previous similar messages [ 1459.121622] LustreError: 91115:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1459.128745] LustreError: 91115:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 5 previous similar messages [ 1462.159634] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1462.160807] LDISKFS-fs (dm-0): recovery complete [ 1462.163291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1462.198158] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1462.303202] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1463.498380] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1464.241746] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1464.244608] Lustre: 94188:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1467.361131] Lustre: 91121:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1467.365026] Lustre: 91121:0:(ldlm_lib.c:2074:extend_recovery_timer()) Skipped 1 previous similar message [ 1467.370840] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1467.387502] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1467.703121] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1468.016970] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1468.020293] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 1 previous similar message [ 1468.335105] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1468.337382] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 1 previous similar message [ 1468.657070] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1468.660656] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) Skipped 8 previous similar messages [ 1469.289107] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1469.291826] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 3 previous similar messages [ 1469.607108] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1469.609541] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 3 previous similar messages [ 1470.863871] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1470.867350] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) Skipped 6 previous similar messages [ 1471.496856] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1471.500152] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 6 previous similar messages [ 1471.815074] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1471.817604] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 6 previous similar messages [ 1474.944868] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1474.948293] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) Skipped 12 previous similar messages [ 1475.577281] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1475.581606] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 12 previous similar messages [ 1475.895085] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1475.897720] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 12 previous similar messages [ 1483.081208] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1483.086521] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) Skipped 25 previous similar messages [ 1483.713113] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1483.716157] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 25 previous similar messages [ 1484.031128] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1484.033964] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 25 previous similar messages [ 1499.224609] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1499.234848] Lustre: 96811:0:(ldlm_lib.c:2074:extend_recovery_timer()) Skipped 48 previous similar messages [ 1500.152310] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1500.162525] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 48 previous similar messages [ 1500.527780] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1500.536436] LustreError: 96811:0:(ldlm_lib.c:2687:replay_request_or_update()) Skipped 48 previous similar messages [ 1524.153361] Lustre: 96811:0:(ldlm_lib.c:2383:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1524.181324] Lustre: 96811:0:(ldlm_lib.c:2393:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1524.210643] Lustre: 96811:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client eb5ffff0-1819-44a1-ac85-80245dbb36ce@192.168.202.25@tcp [ 1524.256565] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1524.284388] Lustre: 96811:0:(ldlm_lib.c:1903:abort_req_replay_queue()) @@@ aborted: req@00000000883ff027 x1856449415283456/t0(8589934773) o101->eb5ffff0-1819-44a1-ac85-80245dbb36ce@192.168.202.25@tcp:41/0 lens 592/0 e 0 to 0 dl 1770448616 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1524.410939] Lustre: lustre-MDT0000: Denying connection for new client eb5ffff0-1819-44a1-ac85-80245dbb36ce (at 192.168.202.25@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1524.457904] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1524.528552] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1524.583483] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1524.594014] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1524.629166] Lustre: Skipped 2 previous similar messages [ 1524.655700] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1524.656700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 1535.970171] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1535.970498] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1535.973967] LustreError: Skipped 1 previous similar message [ 1535.978588] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1535.985924] Lustre: Skipped 4 previous similar messages [ 1535.995856] Lustre: Skipped 7 previous similar messages [ 1550.303334] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1550.364522] Lustre: server umount lustre-OST0000 complete [ 1554.405402] LustreError: 95006:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1554.418307] LustreError: 95006:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 5 previous similar messages [ 1558.530401] Lustre: server umount lustre-OST0001 complete [ 1575.391134] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1575.465849] LustreError: 97874:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1575.468010] LustreError: 97874:0:(obd_class.h:479:obd_check_dev()) Skipped 21 previous similar messages [ 1575.487697] Lustre: server umount lustre-MDT0000 complete [ 1577.178350] LustreError: 94179:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448658 with bad export cookie 4457937863338371537 [ 1577.180618] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1577.183226] LustreError: 94179:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1577.334527] Lustre: server umount lustre-MDT0001 complete [ 1582.092255] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:17:42 (1770448662) [ 1585.824631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1585.829616] Lustre: *** cfs_fail_loc=197, val=0*** [ 1601.311587] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1606.431801] LustreError: 98768:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.125@tcp: failed processing log, type 4: rc = -110 [ 1611.565163] Lustre: *** cfs_fail_loc=197, val=0*** [ 1611.566806] Lustre: Skipped 65 previous similar messages [ 1632.032849] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1632.037356] Lustre: Skipped 7 previous similar messages [ 1634.873993] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1637.562138] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1638.510544] Lustre: server umount lustre-OST0000 complete [ 1642.543874] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:18:43 (1770448723) [ 1658.395036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1658.570595] LustreError: 100274:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1660.259081] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1664.447828] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1666.683701] LustreError: 100308:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1666.689954] LustreError: 100308:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1666.694044] LustreError: 100308:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1666.827490] Lustre: server umount lustre-MDT0000 complete [ 1682.952399] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 1687.443756] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1705.686474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1705.809705] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1705.831242] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1705.873858] Lustre: lustre-MDT0000: new disk, initializing [ 1705.947727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1708.240283] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1714.504881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1714.568833] Lustre: 104326:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1714.726529] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1716.903780] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1721.202330] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1723.530907] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1727.462448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1730.408511] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1734.969201] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1736.886937] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1737.792808] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1738.056864] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1740.253535] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1740.347838] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1742.818095] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1742.819319] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1742.821182] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1742.821188] Lustre: Skipped 15 previous similar messages [ 1742.822498] LustreError: Skipped 1 previous similar message [ 1742.829830] Lustre: Skipped 5 previous similar messages [ 1747.451251] Lustre: server umount lustre-OST0000 complete [ 1755.383264] Lustre: server umount lustre-MDT0000 complete [ 1757.522283] LustreError: 104316:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448839 with bad export cookie 4457937863338382121 [ 1757.524787] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1757.527681] LustreError: 104316:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1757.706991] Lustre: server umount lustre-MDT0001 complete [ 1762.354890] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:20:43 (1770448843) [ 1766.586173] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 1767.969866] Key type lgssc unregistered [ 1768.107402] LNet: 108390:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1768.110987] LNetError: 108390:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1768.121454] LNet: Removed LNI 192.168.202.125@tcp [ 1768.497136] Key type .llcrypt unregistered [ 1768.500871] Key type ._llcrypt unregistered [ 1775.979779] Key type ._llcrypt registered [ 1775.981598] Key type .llcrypt registered [ 1776.044713] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 1784.137523] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1784.755670] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1784.775639] alg: No test for adler32 (adler32-zlib) [ 1785.714702] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 1785.852956] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 1787.471161] Key type lgssc registered [ 1788.206725] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1810.345280] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1810.359191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1811.529523] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1811.558190] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1811.600618] Lustre: lustre-MDT0000: new disk, initializing [ 1811.652347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1811.664060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1813.666251] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1819.739517] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1819.865459] Lustre: lustre-OST0000: new disk, initializing [ 1819.867898] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1819.872541] Lustre: Skipped 1 previous similar message [ 1819.921295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1822.911380] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1827.353319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1827.358768] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1827.375806] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1828.121322] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1828.205798] Lustre: lustre-OST0001: new disk, initializing [ 1828.208961] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1828.244491] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1830.756281] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1834.553278] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1834.558619] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1834.576753] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1839.584921] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1839.585858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1839.589713] Lustre: Skipped 1 previous similar message [ 1843.088068] LustreError: 113921:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 1843.173260] Lustre: server umount lustre-MDT0000 complete [ 1846.326080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1848.733981] LustreError: 111687:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770448930 with bad export cookie 17841522004648789648 [ 1848.736234] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1848.739689] LustreError: 111687:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1848.769325] LustreError: 114515:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1848.773438] LustreError: 114515:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1848.819114] Lustre: server umount lustre-OST0000 complete [ 1850.891345] LustreError: 114716:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1850.895638] LustreError: 114716:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1850.959555] Lustre: server umount lustre-OST0001 complete [ 1854.946972] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 02:22:15 (1770448935) [ 1858.729248] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1869.889239] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:22:30 (1770448950) [ 1889.126888] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 1894.265220] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 1916.453124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1916.642354] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1916.663988] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1916.724257] Lustre: lustre-MDT0000: new disk, initializing [ 1916.771276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1916.794120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1919.172281] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1925.691454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1925.776154] Lustre: 120002:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1925.822986] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1925.826359] Lustre: Skipped 1 previous similar message [ 1925.889160] Lustre: lustre-MDT0001: new disk, initializing [ 1925.950754] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1925.983447] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1925.992437] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1928.513548] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1933.242451] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1935.302163] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1939.875484] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1940.030814] Lustre: lustre-OST0000: new disk, initializing [ 1940.034157] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1940.107051] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1943.858526] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1948.653420] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1950.221453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1950.229522] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1950.304101] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1950.897413] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1951.032712] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1953.410865] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1953.542410] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1955.301505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1955.305729] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1955.310743] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1955.321782] Lustre: Skipped 2 previous similar messages [ 1956.786148] LustreError: 122931:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1956.796525] LustreError: 122931:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1956.861759] Lustre: server umount lustre-OST0000 complete [ 1960.419216] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1960.424910] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1960.427036] Lustre: Skipped 2 previous similar messages [ 1965.137675] LustreError: 123132:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1965.145836] LustreError: 123132:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1965.222338] Lustre: server umount lustre-MDT0000 complete [ 1967.486451] LustreError: 119994:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449049 with bad export cookie 17841522004648790866 [ 1967.490819] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1967.497715] LustreError: 119994:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1967.669326] Lustre: server umount lustre-MDT0001 complete [ 1973.071399] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:24:13 (1770449053) [ 1976.950076] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1977.216922] LustreError: 124003:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1977.233270] LustreError: 124003:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 1977.253976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1979.685851] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1982.435910] LustreError: 124003:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1985.019561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1987.682689] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 1990.672532] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1992.764251] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1996.894758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2000.797975] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2004.221383] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2011.059285] Lustre: *** cfs_fail_loc=159, val=0*** [ 2027.470969] Lustre: lustre-MDT0000: Client 1bfc531d-d829-47aa-8817-814dc26bd79f (at 192.168.202.25@tcp) reconnecting [ 2028.295398] Lustre: *** cfs_fail_loc=159, val=0*** [ 2028.298374] Lustre: Skipped 5 previous similar messages [ 2043.845770] Lustre: lustre-MDT0000: Client 1bfc531d-d829-47aa-8817-814dc26bd79f (at 192.168.202.25@tcp) reconnecting [ 2060.770401] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2060.771445] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2060.771949] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2060.771954] Lustre: Skipped 1 previous similar message [ 2060.775352] Lustre: Skipped 1 previous similar message [ 2060.778531] LustreError: Skipped 1 previous similar message [ 2065.888383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2065.891039] Lustre: Skipped 1 previous similar message [ 2066.224283] LustreError: 126490:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2066.228078] LustreError: 126490:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2066.275671] Lustre: server umount lustre-OST0000 complete [ 2071.008629] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2071.019928] Lustre: Skipped 1 previous similar message [ 2074.480847] Lustre: server umount lustre-MDT0000 complete [ 2076.129096] LustreError: 124897:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2076.139426] LustreError: 124897:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 2076.293829] LustreError: 123983:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449158 with bad export cookie 17841522004648791748 [ 2076.294127] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2076.299831] LustreError: 123983:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2076.471795] Lustre: server umount lustre-MDT0001 complete [ 2081.746542] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2083.162778] Key type lgssc unregistered [ 2083.320241] LNet: 127423:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2083.323514] LNetError: 127423:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2083.336830] LNet: Removed LNI 192.168.202.125@tcp [ 2083.716929] Key type .llcrypt unregistered [ 2083.718191] Key type ._llcrypt unregistered [ 2095.740926] Key type ._llcrypt registered [ 2095.742274] Key type .llcrypt registered [ 2095.810723] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2096.276614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2096.339278] alg: No test for adler32 (adler32-zlib) [ 2097.269224] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2097.411936] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2099.023169] Key type lgssc registered [ 2099.641518] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2103.561235] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2103.575054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2104.861179] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2106.676754] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2110.858113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2111.081937] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2113.087335] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2115.425236] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2116.971715] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2120.036238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2120.183224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2122.760937] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2125.190557] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2129.384059] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2136.803253] Lustre: *** cfs_fail_loc=159, val=0*** [ 2153.416960] Lustre: lustre-MDT0000: Client 9a2af76a-f8c4-48e3-b4be-7bd31030dc82 (at 192.168.202.25@tcp) reconnecting [ 2154.544879] Lustre: *** cfs_fail_loc=159, val=0*** [ 2170.821573] Lustre: lustre-MDT0001: Client 9a2af76a-f8c4-48e3-b4be-7bd31030dc82 (at 192.168.202.25@tcp) reconnecting [ 2171.540810] Lustre: *** cfs_fail_loc=159, val=0*** [ 2171.542764] Lustre: Skipped 3 previous similar messages [ 2187.742461] Lustre: lustre-MDT0001: Client 9a2af76a-f8c4-48e3-b4be-7bd31030dc82 (at 192.168.202.25@tcp) reconnecting [ 2202.979692] Lustre: 131522:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2204.471925] Lustre: *** cfs_fail_loc=159, val=0*** [ 2204.477714] Lustre: Skipped 5 previous similar messages [ 2219.978146] Lustre: lustre-MDT0000: Client 2feb6149-a2d4-4ec1-998a-857bab3b3755 (at 192.168.202.25@tcp) reconnecting [ 2220.773523] Lustre: *** cfs_fail_loc=159, val=0*** [ 2220.775171] Lustre: Skipped 14 previous similar messages [ 2237.363893] Lustre: lustre-MDT0000: Client 2feb6149-a2d4-4ec1-998a-857bab3b3755 (at 192.168.202.25@tcp) reconnecting [ 2238.049393] Lustre: 131813:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2257.381123] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2257.381294] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2257.398574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2257.402891] Lustre: Skipped 1 previous similar message [ 2262.498028] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2262.505841] Lustre: Skipped 1 previous similar message [ 2267.616716] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2267.620950] Lustre: Skipped 1 previous similar message [ 2269.151127] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2269.176165] LustreError: 131939:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2269.237812] Lustre: server umount lustre-OST0000 complete [ 2272.736854] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2272.741187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2272.744477] Lustre: Skipped 2 previous similar messages [ 2272.752607] Lustre: Skipped 1 previous similar message [ 2277.579629] LustreError: 132141:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2277.584359] LustreError: 132141:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2277.623277] Lustre: server umount lustre-MDT0000 complete [ 2277.856687] LustreError: 131578:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2277.863910] LustreError: 131578:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 2279.309972] LustreError: 128777:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449361 with bad export cookie 6478724194967730124 [ 2279.311801] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2279.321109] LustreError: 128777:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2279.406981] LustreError: 132342:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2279.409823] LustreError: 132342:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2279.480975] Lustre: server umount lustre-MDT0001 complete [ 2283.919713] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2285.205561] Key type lgssc unregistered [ 2285.361652] LNet: 132873:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2285.365306] LNetError: 132873:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2285.373855] LNet: Removed LNI 192.168.202.125@tcp [ 2285.746948] Key type .llcrypt unregistered [ 2285.748541] Key type ._llcrypt unregistered [ 2297.191646] Key type ._llcrypt registered [ 2297.193463] Key type .llcrypt registered [ 2297.241945] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2297.705898] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2297.775678] alg: No test for adler32 (adler32-zlib) [ 2298.660691] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2298.758345] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2300.351710] Key type lgssc registered [ 2300.849560] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2304.817591] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2304.830894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2306.048377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2307.959372] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2311.910682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2312.073161] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2313.667181] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2316.146192] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2317.692454] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2320.548271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2320.725972] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2321.770618] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2323.221073] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2325.757562] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2330.987215] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2333.342955] Lustre: 136485:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2334.058000] Lustre: 136534:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2334.594510] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2335.939631] Lustre: 136681:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2336.269228] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2336.272376] Lustre: Skipped 1 previous similar message [ 2336.620286] Lustre: 136779:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2341.346728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2341.350945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2341.357916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2341.361575] Lustre: Skipped 1 previous similar message [ 2343.790130] LustreError: 136876:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2343.834298] Lustre: server umount lustre-OST0000 complete [ 2346.464767] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2346.466369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2346.470288] Lustre: Skipped 2 previous similar messages [ 2351.585730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2351.591161] Lustre: Skipped 2 previous similar messages [ 2351.623993] LustreError: 137085:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2351.626739] LustreError: 137085:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2351.653846] Lustre: server umount lustre-MDT0000 complete [ 2353.208605] LustreError: 134959:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449435 with bad export cookie 9645833521556753208 [ 2353.212123] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2353.213439] LustreError: 134959:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2353.279962] LustreError: 137286:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2353.282778] LustreError: 137286:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2353.339374] Lustre: server umount lustre-MDT0001 complete [ 2357.572520] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2358.736150] Key type lgssc unregistered [ 2358.888875] LNet: 137816:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2358.891828] LNetError: 137816:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2358.902917] LNet: Removed LNI 192.168.202.125@tcp [ 2359.275494] Key type .llcrypt unregistered [ 2359.277538] Key type ._llcrypt unregistered [ 2370.719269] Key type ._llcrypt registered [ 2370.720735] Key type .llcrypt registered [ 2370.764642] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2371.255699] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2371.265377] alg: No test for adler32 (adler32-zlib) [ 2372.132976] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2372.231357] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2373.823171] Key type lgssc registered [ 2374.239445] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2377.477787] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2377.486273] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2378.691301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2380.352042] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2384.323508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2384.502060] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2386.368761] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2388.827786] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2390.442421] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2393.846575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2394.053504] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2395.117264] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2396.708048] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2399.330765] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2403.306275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2407.319043] Lustre: *** cfs_fail_loc=159, val=0*** [ 2422.733981] Lustre: lustre-MDT0000: Client ddbfd23c-42ae-4649-aee8-7f90c04b97dc (at 192.168.202.25@tcp) reconnecting [ 2425.824104] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2425.827657] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2425.833825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2428.900636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2428.900875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2428.907105] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2428.909455] Lustre: Skipped 1 previous similar message [ 2430.639119] LustreError: 141556:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2430.678473] Lustre: server umount lustre-OST0000 complete [ 2434.017186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2434.018359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2434.022514] Lustre: Skipped 1 previous similar message [ 2438.479595] LustreError: 141758:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2438.483350] LustreError: 141758:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2438.536307] Lustre: server umount lustre-MDT0000 complete [ 2439.142264] LustreError: 139188:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2439.149165] LustreError: 139188:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 2440.404028] LustreError: 141759:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449522 with bad export cookie 9607771834592078693 [ 2440.406630] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2440.409859] LustreError: 141759:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2440.495756] LustreError: 141960:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2440.499057] LustreError: 141960:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2440.598171] Lustre: server umount lustre-MDT0001 complete [ 2445.501650] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2446.758226] Key type lgssc unregistered [ 2446.907542] LNet: 142491:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2446.911306] LNetError: 142491:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2446.926539] LNet: Removed LNI 192.168.202.125@tcp [ 2447.303993] Key type .llcrypt unregistered [ 2447.305486] Key type ._llcrypt unregistered [ 2458.296618] Key type ._llcrypt registered [ 2458.298388] Key type .llcrypt registered [ 2458.354619] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2458.763263] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2458.840094] alg: No test for adler32 (adler32-zlib) [ 2459.737333] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2459.845094] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2461.479497] Key type lgssc registered [ 2461.983478] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2465.505287] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2465.517880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2466.754874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2468.565930] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2472.600654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2472.777299] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2474.610897] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2477.234497] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2479.259872] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2482.673173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2482.892932] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2483.944067] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2485.974756] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2488.526676] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2492.143663] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2507.139242] Lustre: lustre-OST0000: evicting 3efd6481-f383-4ce6-8bb6-ed034d6014c2 (at 192.168.202.25@tcp) by administrative request [ 2507.143649] Lustre: Skipped 1 previous similar message [ 2533.346597] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2533.346697] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2533.347306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2533.356925] Lustre: Skipped 1 previous similar message [ 2538.465755] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2538.468907] Lustre: Skipped 2 previous similar messages [ 2543.585458] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2543.587721] Lustre: Skipped 1 previous similar message [ 2546.143158] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2546.166335] LustreError: 146639:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2546.221088] Lustre: server umount lustre-OST0000 complete [ 2548.706909] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2548.712747] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2548.716649] Lustre: Skipped 1 previous similar message [ 2553.824369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2553.828507] Lustre: Skipped 1 previous similar message [ 2554.117778] LustreError: 146842:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2554.121191] LustreError: 146842:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2554.159813] Lustre: server umount lustre-MDT0000 complete [ 2555.910366] LustreError: 146640:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449637 with bad export cookie 638777514631647210 [ 2555.913394] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2555.915114] LustreError: 146640:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2555.990993] LustreError: 147044:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2555.993295] LustreError: 147044:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2556.056804] Lustre: server umount lustre-MDT0001 complete [ 2560.431245] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2561.642873] Key type lgssc unregistered [ 2561.789837] LNet: 147576:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2561.793877] LNetError: 147576:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2561.807435] LNet: Removed LNI 192.168.202.125@tcp [ 2562.174387] Key type .llcrypt unregistered [ 2562.175633] Key type ._llcrypt unregistered [ 2593.687041] Key type ._llcrypt registered [ 2593.689126] Key type .llcrypt registered [ 2593.742463] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2594.245828] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2594.256274] alg: No test for adler32 (adler32-zlib) [ 2595.144849] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2595.265400] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2596.895230] Key type lgssc registered [ 2597.405708] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2600.613927] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2600.622967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2601.801603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2603.250918] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2606.455399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2606.620729] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2608.144568] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2610.256451] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2611.761892] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2614.667743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2614.816314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2616.872716] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2617.094495] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2619.333021] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2628.134511] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2633.186343] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2633.187215] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2633.187624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2633.192789] Lustre: Skipped 1 previous similar message [ 2635.054101] LustreError: 152292:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2635.094944] Lustre: server umount lustre-OST0000 complete [ 2637.282093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2637.285233] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2637.291683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2637.294911] Lustre: Skipped 1 previous similar message [ 2638.304435] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2638.305138] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2638.782791] LustreError: 152492:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2638.785457] LustreError: 152492:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2638.812312] Lustre: server umount lustre-MDT0000 complete [ 2640.309593] LustreError: 150000:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449722 with bad export cookie 16904293593665702126 [ 2640.313718] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2640.314218] LustreError: 150000:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2640.379754] LustreError: 152693:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2640.382463] LustreError: 152693:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2640.443302] Lustre: server umount lustre-MDT0001 complete [ 2644.272207] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2645.347601] Key type lgssc unregistered [ 2645.492442] LNet: 153223:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2645.495229] LNetError: 153223:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2645.505287] LNet: Removed LNI 192.168.202.125@tcp [ 2645.809459] Key type .llcrypt unregistered [ 2645.811378] Key type ._llcrypt unregistered [ 2656.152181] Key type ._llcrypt registered [ 2656.153293] Key type .llcrypt registered [ 2656.196883] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2656.539706] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2656.576033] alg: No test for adler32 (adler32-zlib) [ 2657.438410] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2657.527417] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2659.111207] Key type lgssc registered [ 2659.532425] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2665.148507] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:35:46 (1770449746) [ 2671.111287] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 2674.368824] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2687.633491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2687.639888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2688.739408] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2688.752877] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2688.792555] Lustre: lustre-MDT0000: new disk, initializing [ 2688.818731] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2688.826197] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2690.262019] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2694.731512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2694.777937] Lustre: 157401:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2694.802871] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2694.805080] Lustre: Skipped 1 previous similar message [ 2694.841665] Lustre: lustre-MDT0001: new disk, initializing [ 2694.878486] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2694.891595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2694.896587] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2696.390393] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2699.828519] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2701.197607] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2703.703071] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2703.829587] Lustre: lustre-OST0000: new disk, initializing [ 2703.832024] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2703.870755] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2705.948489] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2709.253991] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2711.035336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2711.038887] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2711.074490] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2721.249711] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2721.251208] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2721.258739] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2721.260810] Lustre: Skipped 1 previous similar message [ 2724.974121] LustreError: 159982:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2725.023750] Lustre: server umount lustre-OST0000 complete [ 2730.975876] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2730.979062] LustreError: Skipped 1 previous similar message [ 2730.980647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2730.985062] Lustre: Skipped 1 previous similar message [ 2730.987484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2732.544859] LustreError: 160184:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2732.547557] LustreError: 160184:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2732.578810] Lustre: server umount lustre-MDT0000 complete [ 2733.888192] LustreError: 158180:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770449815 with bad export cookie 16345414285160897819 [ 2733.891305] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2733.892658] LustreError: 158180:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2733.960052] LustreError: 160385:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2733.962227] LustreError: 160385:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2734.018209] Lustre: server umount lustre-MDT0001 complete [ 2737.736035] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 2738.857978] Key type lgssc unregistered [ 2739.000324] LNet: 160916:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2739.004517] LNetError: 160916:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2739.012851] LNet: Removed LNI 192.168.202.125@tcp [ 2739.360155] Key type .llcrypt unregistered [ 2739.361622] Key type ._llcrypt unregistered [ 2749.123282] Key type ._llcrypt registered [ 2749.124939] Key type .llcrypt registered [ 2749.166854] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 2749.452778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2749.460396] alg: No test for adler32 (adler32-zlib) [ 2750.329128] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 2750.419526] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 2752.007152] Key type lgssc registered [ 2752.382071] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2755.070201] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2755.077547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2756.239248] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2757.643693] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2760.886335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2761.025723] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2762.521808] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2764.509792] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2765.799268] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2768.391250] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2768.530749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2770.725700] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 2772.977283] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2782.889447] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2786.129070] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2786.138501] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2786.143656] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2786.147771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.152613] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2786.507951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.873075] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2786.875844] Lustre: Skipped 1 previous similar message [ 2786.882080] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2786.887301] Lustre: Skipped 1 previous similar message [ 2786.889261] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2786.891850] Lustre: Skipped 1 previous similar message [ 2786.893784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.898637] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2786.901067] Lustre: Skipped 1 previous similar message [ 2787.219047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.551233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.860532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.169482] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2788.171814] Lustre: Skipped 3 previous similar messages [ 2788.177218] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2788.181325] Lustre: Skipped 3 previous similar messages [ 2788.182904] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2788.185544] Lustre: Skipped 3 previous similar messages [ 2788.186902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.190694] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2788.193050] Lustre: Skipped 3 previous similar messages [ 2788.496207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.820475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.135073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.482674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.799995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.121242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.456833] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2790.459252] Lustre: Skipped 6 previous similar messages [ 2790.465422] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2790.471548] Lustre: Skipped 6 previous similar messages [ 2790.474102] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2790.476725] Lustre: Skipped 6 previous similar messages [ 2790.479091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.483928] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2790.488038] Lustre: Skipped 6 previous similar messages [ 2790.804376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.109338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.426548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.736673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.070250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.399113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.746521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.065285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.378662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.686217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.002373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.312757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.647550] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2794.650267] Lustre: Skipped 12 previous similar messages [ 2794.657383] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2794.662166] Lustre: Skipped 12 previous similar messages [ 2794.664393] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2794.666899] Lustre: Skipped 12 previous similar messages [ 2794.668893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.673162] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2794.675767] Lustre: Skipped 12 previous similar messages [ 2795.005228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.379480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.724312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.091055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.444623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.787739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.111205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.449581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.800592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.172051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.511139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.851661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.171829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.523074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.863895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.220394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.557951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.919044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.258104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.600739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.937398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.262224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.576030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.934493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2802.937724] Lustre: Skipped 23 previous similar messages [ 2802.945968] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2802.951469] Lustre: Skipped 23 previous similar messages [ 2802.953922] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2802.957676] Lustre: Skipped 23 previous similar messages [ 2802.959332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.963889] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2802.969150] Lustre: Skipped 23 previous similar messages [ 2803.316272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.669246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.059926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.402114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.747099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.071623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.413316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.729324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.061748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.405391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.748922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.080393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.413496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.760501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.114269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.429800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.768707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.121283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.458965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.808612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.153655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.501255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.823846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.154833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.500647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.843547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.178820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.486626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.799700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.148536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.524648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.883368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.218431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.540088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.859670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.195270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.503640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.829864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.148175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.504823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.838988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.195694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.515944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.849413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.188061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.541290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.894516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.233682] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2819.236486] Lustre: Skipped 47 previous similar messages [ 2819.242941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2819.248021] Lustre: Skipped 47 previous similar messages [ 2819.250171] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2819.252535] Lustre: Skipped 47 previous similar messages [ 2819.254126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.257979] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2819.260309] Lustre: Skipped 47 previous similar messages [ 2819.602316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.951124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.260349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.580653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.896060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.211444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.523125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.851790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.172044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.474409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.775246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.089614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.414256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.731908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.067175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.410469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.765770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.098822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.409539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.707774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.046242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.386036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.717813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.047630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.395463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.730545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.047705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.374913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.699484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.020193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.335890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.696916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.041094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.395843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.766341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.136596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.490170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.824768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.149159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.471219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.804463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.129156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.447831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.800504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.146992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.481309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.837893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.207887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.564887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.907497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.241719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.614957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.980402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.327579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.683556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.017669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.354284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.698332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.072584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.415791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.772424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.126041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.473768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.833244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.163869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.504843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.874654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.198977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.569562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.897444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.204177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.531151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.855531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.158390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.515147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.826581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.175754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.531188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.865202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.177652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.512538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.840848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.171982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.492014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.838674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.167469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.491622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.864499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.210591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.551814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.906517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.285646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.639457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.977431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.307800] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2851.313515] Lustre: Skipped 94 previous similar messages [ 2851.320761] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2851.324776] Lustre: Skipped 94 previous similar messages [ 2851.326632] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2851.329068] Lustre: Skipped 94 previous similar messages [ 2851.330844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.336424] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2851.338619] Lustre: Skipped 94 previous similar messages [ 2851.659642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.992690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.311388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.646797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.008487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.332984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.665714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.012665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.349502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.660653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.984254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.324225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.641810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.975277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.301632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.647229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.985156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.299552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.613689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.944344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.250070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.556452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.875640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.208402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.529692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.840067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.150549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.462476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.761261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.061436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.359781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.697158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.040721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.383750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.726115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.068978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.410489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.740751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.062739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.384634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.712605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.094962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.463891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.823410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.212885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.543272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.874193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.230248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.594038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.949953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.309303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.661781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.005564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.337741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.671673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.055792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.461630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.845052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.200248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.558662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.930322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.271345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.623595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.931773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.252081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.574371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.908050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.219382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.533616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.839715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.146544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.456696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.784425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.104653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.428704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.740010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.074392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.375035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.667867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.961493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.277890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.565143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.850128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.140946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.446707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.739953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.046364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.331600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.636157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.937806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.239098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.540838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.870552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.185294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.514434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.833370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.167844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.471769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.789135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.115303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.473049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.807644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.153902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.485233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.837285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.179119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.520123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.865996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.207108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.554130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.909373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.237052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.585955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.917348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.235357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.534609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.869429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.170929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.483374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.793969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.156843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.457815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.788826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.125046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.436069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.745669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.063433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.362608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.701388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.019026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.343321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.682640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.023532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.369718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.711375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.045183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.401512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.731267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.078318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.408183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.738674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.069958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.413934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.739601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.061218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.366592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.698482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.016150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.348688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.673853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.027779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.372481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.681320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.992447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.316491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.619881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.935823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.260228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.587327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.889045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.213970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.526646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.851648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.176759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.496628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.811785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.163866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.466800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.766279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.059099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.355533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.645939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.953428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.279977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.617622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.937908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.241115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.551290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.876407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.191564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.508577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.845879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.222195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.559789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.904826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.235247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.590068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.963516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.356384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.708882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.096670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.456209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.821749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.177113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.510214] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2915.513181] Lustre: Skipped 194 previous similar messages [ 2915.519683] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2915.524594] Lustre: Skipped 194 previous similar messages [ 2915.526681] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2915.529182] Lustre: Skipped 194 previous similar messages [ 2915.531032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.534956] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2915.537145] Lustre: Skipped 194 previous similar messages [ 2915.867210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.223398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.576304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.940956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.302872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.652108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.029443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.383975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.790636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.165178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.520994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.894882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.225382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.563088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.921944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.311857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.727712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.142880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.568344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.926079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.277141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.653912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.021352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.357150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.701746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.039982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.367794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.712812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.049195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.370806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.713823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.059975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.397917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.719453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.040256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.363602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.739446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.102844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.473553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.824466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.161045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.503979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.837720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.208585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.534629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.882936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.225068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.557169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.890612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.214991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.573874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.910236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.289223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.630712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.988985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.332832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.676256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.003739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.347348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.678907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.062232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.435238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.799052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.161813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.527388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.887750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.269520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.618850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.956624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.283679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.626986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.978073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.311171] LustreError: 162308:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2941.316264] LustreError: 162308:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2941.319137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.668992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.020034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.403328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.748036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.080880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.443131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.785069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.109374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.455112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.814608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.168179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.522622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.880132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.232343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.629327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.021112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.437903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.885225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.300465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.640119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.005789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.368952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.757688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.146800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.522102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.892947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.269176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.661736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.015221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.375268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.731767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.067236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.445113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.790814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.137580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.471438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.820552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.154310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.496570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.833839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.177416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.509649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.843053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.172901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.504336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.852285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.227524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.585963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.960731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.380575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.757936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.145304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.546502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.918589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.262638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.612775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.973817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.314946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.649377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.025281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.365281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.708134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.060803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.420841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.774474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.123083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.464632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.793556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.145816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.492846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.868175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.198231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.599846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.973589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.343299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.699465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.044727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.386606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.784447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.181835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.564671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.941853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.467131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.840289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.192153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.561609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.929253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.316379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.683399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.054383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.390922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.732820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.060818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.394505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.779659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.162352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.514908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.878910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.242527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.596483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.940197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.282835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.641761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.982464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.316920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.661642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.997490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.344608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.738257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.107716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.469420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.810189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.189520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.531573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.854946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.265727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.653110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.995645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.366684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.693502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.012876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.356778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.724099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.064707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.429728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.803638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.222804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.579084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.952092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.300350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.668899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.034539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.394212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.774386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.151337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.517916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.863039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.201815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.539162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.864145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.173669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.529722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.886894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.214630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.573436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.919762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.285868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.645450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.009814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.368811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.709329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.066428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.415857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.750275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.077554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.399917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.738182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.090661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.433491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.779834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.155463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.536027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.920334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.290835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.629070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.990824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.365901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.705206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.034145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.368165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.713415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.087617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.446752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.878911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.281768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.697286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.078591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.441761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.819837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.155595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.513073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.846618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.181834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.521416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.886653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.271639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.661032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.014780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.376106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.744028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.143802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.470234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.825868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.183717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.560090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.956618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.329030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.683278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.054913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.414910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.766722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.123205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.510774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.867432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.229685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.592617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.980708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.342048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.737996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.075336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.423980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.794434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.226330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.592605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.967651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.377908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.778183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.160768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.523321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.877285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.225018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.628391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.051070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.414473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.785149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.143334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.498415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.841238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.199927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.557846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.888075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.229991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.555130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.876564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.207244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.547676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.899722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.291392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.657262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.998879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.365592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.703641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.045228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.402273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.792794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.120579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.453514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.771342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.157652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.513679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.878119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.233668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.596872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.945272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.294867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.627503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.950343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.299320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.692879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.029470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.354255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.676748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.008698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.328654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.632891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.962945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.301342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.613298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.931908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.241266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.572762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.887829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.184210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.484511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.807499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.095107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.391725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.694369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.015231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.323619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.639479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.953260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.268408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.567906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.869685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.171229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.490713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.780393] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3043.782364] Lustre: Skipped 360 previous similar messages [ 3043.787285] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3043.790404] Lustre: Skipped 360 previous similar messages [ 3043.791777] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3043.793964] Lustre: Skipped 360 previous similar messages [ 3043.795994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.799899] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3043.802439] Lustre: Skipped 360 previous similar messages [ 3044.108664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.419168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.722228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.009169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.298489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.596358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.911878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.203981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.488900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.776598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.069107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.357277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.648911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.967226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.298426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.601520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.920597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.226714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.540420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.850545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.175726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.473544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.805889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.139826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.449672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.792098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.129256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.453686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.780519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.090341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.411504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.720477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.020231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.333109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.663464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.977217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.287475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.594273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.912834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.229128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.537078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.844273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.160563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.461057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.773536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.073583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.395974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.707156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.025064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.315531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.624555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.966370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.299303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.612463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.926655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.234071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.557846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.850630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.181636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.487639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.782650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.068434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.358531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.659875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.984177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.314719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.654161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.954809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.266142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.570213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.890082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.198732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.496083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.793262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.103441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.417200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.760387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.102407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.456360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.786530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.143488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.491264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.822811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.142675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.442201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.737082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.034506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.331470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.645840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.951343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.277428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.602078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.956881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.271169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.591255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.897850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.215254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.522505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.818338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.138853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.479260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.787525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.106219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.416066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.731897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.052336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.389925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.716197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.043820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.379445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.726487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.049634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.354098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.650401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.975974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.282905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.609634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.962468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.297645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.612406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.932573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.246368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.559674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.871457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.197966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.479690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.785042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.087164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.393043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.704901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.042164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.366435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.696872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.012563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.340873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.675281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.019478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.362803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.686546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.988517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.299610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.606035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.944474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.272483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.620324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.948728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.263285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.566062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.887810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.216430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.568718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.918144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.257335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.607852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.959054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.326729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.689100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.048717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.381348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.728686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.079384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.439049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.783699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.119993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.468362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.806265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.128544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.461709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.822053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.147399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.496919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.835532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.207076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.533885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.869210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.178204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.513114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.840253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.166902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.482647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.827933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.145563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.438514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.748968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.067014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.366054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.676526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.974301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.311761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.612357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.915789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.211234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.512583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.790721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.092846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.422960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.771318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.095599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.425591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.752239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.102146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.430305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.754475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.090192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.446928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.739991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.066782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.380254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.702476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.043010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.389288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.699912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.026100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.357369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.679144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.985248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.319170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.626744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.934994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.257648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.590023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.880893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.194329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.510188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.844000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.173630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.474736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.801344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.117886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.419114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.745576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.049741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.362235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.667742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.969539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.278568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.639927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.979496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.309119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.636241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.958905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.265278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.580118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.883765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.198993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.501965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.820071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.118481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.433638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.751092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.271124] LustreError: 164193:0:(ofd_dev.c:1830:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3199.274373] LustreError: 162308:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3262.431908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3262.435541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3264.481955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3264.483566] Lustre: Skipped 1 previous similar message [ 3268.019120] LustreError: 216035:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3268.094700] Lustre: server umount lustre-OST0000 complete [ 3269.602114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3269.604757] Lustre: Skipped 1 previous similar message [ 3274.720890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3274.723303] Lustre: Skipped 1 previous similar message [ 3275.454904] LustreError: 216236:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3275.458131] LustreError: 216236:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3275.524560] Lustre: server umount lustre-MDT0000 complete [ 3276.665052] LustreError: 216237:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450358 with bad export cookie 17641083989515241459 [ 3276.667494] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3276.668833] LustreError: 216237:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3276.735064] LustreError: 216439:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3276.736855] LustreError: 216439:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3276.783429] Lustre: server umount lustre-MDT0001 complete [ 3280.224394] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 3281.177783] Key type lgssc unregistered [ 3281.306338] LNet: 216970:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3281.308996] LNetError: 216970:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3281.317299] LNet: Removed LNI 192.168.202.125@tcp [ 3281.599536] Key type .llcrypt unregistered [ 3281.600657] Key type ._llcrypt unregistered [ 3290.295376] Key type ._llcrypt registered [ 3290.296851] Key type .llcrypt registered [ 3290.328418] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3290.628087] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3290.684756] alg: No test for adler32 (adler32-zlib) [ 3291.528486] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 3291.606618] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 3293.183161] Key type lgssc registered [ 3293.517187] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3296.029509] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3296.036841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3297.186448] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3298.410030] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3301.152464] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3301.284733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3302.577462] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3304.177649] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3305.303782] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3307.503104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3307.613889] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3309.375824] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3311.006353] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3321.957861] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3328.222048] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3328.781767] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3328.786449] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3328.789270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.795122] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3328.795130] LustreError: 218360:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3331.511432] Lustre: lustre-OST0000: Not available for connect from 192.168.202.25@tcp (stopping) [ 3332.576635] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3332.577610] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3332.578238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3332.580477] Lustre: Skipped 1 previous similar message [ 3335.467167] LustreError: 220755:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3335.501861] Lustre: server umount lustre-OST0000 complete [ 3348.871136] LustreError: 218360:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3363.541024] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3363.656765] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3365.513012] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3367.198215] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3372.898656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.902147] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3372.902191] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.823952] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 6 sec [ 3374.931069] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa004071d5800.ost_server_uuid 50 [ 3375.450641] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa004071d5800.ost_server_uuid in FULL state after 0 sec [ 3378.145332] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3378.147634] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3378.152483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3378.155345] Lustre: Skipped 2 previous similar messages [ 3382.378423] LustreError: 222070:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3382.380746] LustreError: 222070:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3382.417131] Lustre: server umount lustre-OST0000 complete [ 3388.384669] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3388.384821] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3388.391924] Lustre: Skipped 1 previous similar message [ 3388.394605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3388.396537] Lustre: Skipped 1 previous similar message [ 3393.504038] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3393.507085] Lustre: Skipped 1 previous similar message [ 3398.111133] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3398.148098] LustreError: 222272:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3398.151750] LustreError: 222272:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3398.181146] Lustre: server umount lustre-MDT0000 complete [ 3398.624276] LustreError: 220487:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3398.630473] LustreError: 220487:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 3399.392335] LustreError: 218322:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450481 with bad export cookie 2410411771179607132 [ 3399.395224] LustreError: 218322:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3399.396301] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3399.528044] Lustre: server umount lustre-MDT0001 complete [ 3402.798769] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 3403.860453] Key type lgssc unregistered [ 3403.981421] LNet: 223014:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3403.985891] LNetError: 223014:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3403.993330] LNet: Removed LNI 192.168.202.125@tcp [ 3404.294077] Key type .llcrypt unregistered [ 3404.295663] Key type ._llcrypt unregistered [ 3416.031139] Key type ._llcrypt registered [ 3416.032277] Key type .llcrypt registered [ 3416.078367] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3416.587746] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3416.659707] alg: No test for adler32 (adler32-zlib) [ 3417.508092] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 3417.583925] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 3419.159110] Key type lgssc registered [ 3419.534440] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3422.524315] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3422.531123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3423.677596] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3424.971966] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3427.896395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3428.033215] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3429.328383] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3430.323647] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3432.546138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3432.678091] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3433.709298] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3434.891770] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3437.839948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3437.905712] Lustre: lustre-OST0001: new disk, initializing [ 3437.908483] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3437.944926] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3439.943484] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3442.178563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3442.182498] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3442.221931] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3445.293380] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3447.794223] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3462.626037] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3462.628518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3462.630270] Lustre: Skipped 2 previous similar messages [ 3462.632604] Lustre: Skipped 1 previous similar message [ 3465.409815] LustreError: 228054:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3465.451189] Lustre: server umount lustre-MDT0000 complete [ 3466.708426] LustreError: 224681:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450548 with bad export cookie 2573955540217530005 [ 3466.711108] LustreError: 225419:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 3466.712526] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3466.714421] LustreError: 224681:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3466.785736] LustreError: 228256:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3466.787557] LustreError: 228256:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3466.837798] Lustre: server umount lustre-MDT0001 complete [ 3477.366101] LustreError: 228456:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3477.368648] LustreError: 228456:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3477.386177] Lustre: server umount lustre-OST0000 complete [ 3488.819024] LustreError: 228658:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3488.821108] LustreError: 228658:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3488.864130] Lustre: server umount lustre-OST0001 complete [ 3495.683293] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3499.287486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3499.440160] LustreError: 230035:0:(ldlm_lib.c:1185:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3499.448452] LustreError: 230035:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 2 previous similar messages [ 3499.466400] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3500.793148] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3503.536728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3504.943512] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3505.879027] Lustre: 231141:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3508.100487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3508.242258] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3508.244385] Lustre: Skipped 1 previous similar message [ 3509.287809] LustreError: 231497:0:(ldlm_lib.c:1185:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3509.291449] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 3510.112562] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3513.153845] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3514.283616] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 3514.286502] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 3515.094112] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3518.311325] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3519.576279] Lustre: 232983:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3532.092476] LustreError: 233292:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3532.094890] LustreError: 233292:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3532.120578] Lustre: server umount mylustre-MDT0000 complete [ 3533.334107] LustreError: 230011:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450615 with bad export cookie 2573955540217532868 [ 3533.341228] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3533.461825] Lustre: server umount mylustre-MDT0001 complete [ 3551.199124] Lustre: 223963:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1770450616/real 1770450616] req@ffff904be3be3b80 x1856451902627712/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1770450632 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 3551.199195] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3551.206737] Lustre: 223963:0:(client.c:2478:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 3551.210852] Lustre: Skipped 1 previous similar message [ 3554.271248] Lustre: 223966:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1770450620/real 1770450620] req@ffff904bcd143480 x1856451902628096/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1770450636 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 3554.314172] LustreError: 233693:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3554.317431] LustreError: 233693:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 3554.338175] Lustre: server umount mylustre-OST0000 complete [ 3555.689112] Lustre: server umount mylustre-OST0001 complete [ 3562.715918] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3566.242205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3566.397560] LustreError: 235273:0:(ldlm_lib.c:1185:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3566.419750] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3566.423404] Lustre: Skipped 1 previous similar message [ 3567.665342] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3570.468398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3571.890323] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3572.861824] Lustre: 236384:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3575.091522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3576.228438] LustreError: 236738:0:(ldlm_lib.c:1185:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3576.232410] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 3577.131779] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3580.014872] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3581.164773] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 3581.164830] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 3581.949721] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3585.062241] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3586.213852] Lustre: 238222:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3596.256141] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3596.258342] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3596.261842] Lustre: Skipped 1 previous similar message [ 3596.263559] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3596.265166] Lustre: Skipped 2 previous similar messages [ 3599.425068] LustreError: 238472:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3599.427011] LustreError: 238472:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3599.457730] Lustre: server umount tfs-MDT0000 complete [ 3600.698036] LustreError: 235248:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450682 with bad export cookie 2573955540217535052 [ 3600.701142] LustreError: 235248:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 3600.701407] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3600.811037] Lustre: server umount tfs-MDT0001 complete [ 3611.264622] Lustre: server umount tfs-OST0000 complete [ 3622.809750] Lustre: server umount tfs-OST0001 complete [ 3629.751204] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3633.227068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3633.410200] LustreError: 240454:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3633.436897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3633.439288] Lustre: Skipped 3 previous similar messages [ 3634.758848] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3637.697024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3639.202732] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3640.219691] Lustre: 241560:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3642.287029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3643.426387] LustreError: 241915:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3643.431100] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 3644.238924] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3647.041265] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3648.170184] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 3648.170240] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 3648.949587] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3651.888711] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3652.956152] Lustre: 243403:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3660.103026] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:52:21 (1770450741) [ 3673.567797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3673.570123] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3673.573959] Lustre: Skipped 3 previous similar messages [ 3673.575745] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3673.578372] Lustre: Skipped 3 previous similar messages [ 3679.203678] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3679.205661] Lustre: Skipped 7 previous similar messages [ 3684.321229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3686.367147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3686.403591] LustreError: 243770:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3686.405463] LustreError: 243770:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3686.435057] Lustre: server umount lustre-MDT0000 complete [ 3687.708424] LustreError: 241180:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450769 with bad export cookie 2573955540217536977 [ 3687.713714] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3689.440108] LustreError: 241168:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3689.440837] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3689.445510] LustreError: 241168:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 3689.450030] Lustre: Skipped 4 previous similar messages [ 3692.448565] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3692.450513] Lustre: Skipped 6 previous similar messages [ 3702.239125] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3702.318081] Lustre: server umount lustre-MDT0001 complete [ 3709.694624] Lustre: server umount lustre-OST0000 complete [ 3725.279131] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3725.333490] Lustre: server umount lustre-OST0001 complete [ 3727.404373] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 3730.748485] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3743.169958] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3743.169958] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3743.169958] [ 3743.176393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3743.269574] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3743.281602] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3743.319831] Lustre: lustre-MDT0000: new disk, initializing [ 3743.346033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3743.348851] Lustre: Skipped 3 previous similar messages [ 3743.355557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3744.702481] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3748.854706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3748.895255] Lustre: 246546:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3748.920680] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3748.923333] Lustre: Skipped 1 previous similar message [ 3748.960236] Lustre: lustre-MDT0001: new disk, initializing [ 3749.009986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3749.013547] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3750.325109] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3754.519926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3754.629298] Lustre: lustre-OST0000: new disk, initializing [ 3754.631280] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3756.419893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3756.425334] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3756.468403] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3756.611900] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3760.744191] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3760.810239] Lustre: lustre-OST0001: new disk, initializing [ 3760.813269] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3762.627898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3762.631832] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3762.667662] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3762.878079] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3767.400609] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:54:08 (1770450848) [ 3770.821932] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:54:11 (1770450851) [ 3783.143373] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3783.149149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3783.150926] Lustre: Skipped 3 previous similar messages [ 3788.205193] LustreError: 249960:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3788.207479] LustreError: 249960:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3788.228634] Lustre: server umount lustre-OST0000 complete [ 3788.259559] LustreError: 248081:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3788.266510] LustreError: 248081:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 5 previous similar messages [ 3790.304142] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3795.690145] Lustre: server umount lustre-MDT0000 complete [ 3796.967359] LustreError: 246539:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450878 with bad export cookie 2573955540217538755 [ 3796.971736] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3796.972529] LustreError: 246539:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3811.295152] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3811.382047] Lustre: server umount lustre-MDT0001 complete [ 3815.035316] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 3824.790649] Lustre: server umount lustre-OST0001 complete [ 3826.881232] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 3833.373176] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3845.736703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3845.827768] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3845.839586] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3845.881470] Lustre: lustre-MDT0000: new disk, initializing [ 3845.911616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3847.194367] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3851.408974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3851.454155] Lustre: 253410:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3851.576301] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3852.875561] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3856.009436] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3857.165513] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3859.448940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3861.308528] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3861.344174] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3861.451752] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3864.515207] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3871.712586] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3871.717226] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3871.718965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3871.721585] Lustre: Skipped 6 previous similar messages [ 3871.723213] Lustre: Skipped 7 previous similar messages [ 3872.987267] Lustre: server umount lustre-OST0000 complete [ 3880.472235] Lustre: server umount lustre-MDT0000 complete [ 3881.743159] LustreError: 253400:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770450963 with bad export cookie 2573955540217540225 [ 3881.745190] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3881.747188] LustreError: 253400:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3881.867200] Lustre: server umount lustre-MDT0001 complete [ 3885.518449] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 3886.542315] Key type lgssc unregistered [ 3886.668369] LNet: 256780:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3886.670730] LNetError: 256780:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3886.681308] LNet: Removed LNI 192.168.202.125@tcp [ 3886.988116] Key type .llcrypt unregistered [ 3886.989401] Key type ._llcrypt unregistered [ 3896.598797] Key type ._llcrypt registered [ 3896.599832] Key type .llcrypt registered [ 3896.635328] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3897.040752] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3897.049170] alg: No test for adler32 (adler32-zlib) [ 3897.905224] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 3897.985969] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 3899.567138] Key type lgssc registered [ 3899.928170] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3902.467711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3902.473371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3903.624535] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3904.949418] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3907.849193] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3907.983438] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3909.348782] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3911.202809] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3912.362540] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3914.656526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3914.788461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3916.631553] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 3918.405437] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3918.771179] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3922.913452] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3922.916077] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3922.920693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3925.484091] LustreError: 260484:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3925.519225] Lustre: server umount lustre-OST0000 complete [ 3928.033803] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3928.034477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3928.037320] Lustre: Skipped 1 previous similar message [ 3928.040286] Lustre: Skipped 1 previous similar message [ 3933.152197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3933.154919] Lustre: Skipped 1 previous similar message [ 3938.272203] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3938.274366] Lustre: Skipped 1 previous similar message [ 3941.343120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3941.377516] LustreError: 260690:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3941.381288] LustreError: 260690:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3941.411995] Lustre: server umount lustre-MDT0000 complete [ 3942.368780] LustreError: 260490:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3942.374045] LustreError: 260490:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 3942.712414] LustreError: 258179:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451024 with bad export cookie 8937380090848090229 [ 3942.716129] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3942.717757] LustreError: 258179:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3942.784926] LustreError: 260894:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3942.787366] LustreError: 260894:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3942.836697] Lustre: server umount lustre-MDT0001 complete [ 3946.393406] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 3947.447416] Key type lgssc unregistered [ 3947.575416] LNet: 261425:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3947.577602] LNetError: 261425:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3947.585333] LNet: Removed LNI 192.168.202.125@tcp [ 3947.875259] Key type .llcrypt unregistered [ 3947.876451] Key type ._llcrypt unregistered [ 3954.445343] Key type ._llcrypt registered [ 3954.446669] Key type .llcrypt registered [ 3954.483435] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 3954.805551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3954.814931] alg: No test for adler32 (adler32-zlib) [ 3955.684139] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 3955.778050] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 3957.359152] Key type lgssc registered [ 3957.728236] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3960.779341] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3960.784464] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3976.223337] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3981.343208] LustreError: 262576:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.125@tcp: failed processing log, type 4: rc = -110 [ 4007.967145] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4009.878237] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4011.658743] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4012.327937] LustreError: 263237:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 4012.372043] Lustre: server umount lustre-OST0000 complete [ 4017.728557] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 4018.734670] Key type lgssc unregistered [ 4018.866358] LNet: 264072:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4018.868880] LNetError: 264072:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4018.877297] LNet: Removed LNI 192.168.202.125@tcp [ 4019.157347] Key type .llcrypt unregistered [ 4019.158396] Key type ._llcrypt unregistered [ 4032.546045] Key type ._llcrypt registered [ 4032.547294] Key type .llcrypt registered [ 4032.590705] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4032.909058] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4032.953458] alg: No test for adler32 (adler32-zlib) [ 4033.817865] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 4033.907593] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 4035.487231] Key type lgssc registered [ 4035.893801] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4040.554101] loop0: detected capacity change from 0 to 204800000 [ 4040.666783] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4040.670849] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4040.785794] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4041.147154] loop0: detected capacity change from 0 to 204800000 [ 4041.206767] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4041.210413] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4041.336064] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4041.670539] loop0: detected capacity change from 0 to 204800000 [ 4041.734145] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4041.737848] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4041.778760] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4042.141799] loop0: detected capacity change from 0 to 204800000 [ 4042.203567] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4042.626114] loop0: detected capacity change from 0 to 204800000 [ 4042.641788] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4044.013272] loop0: detected capacity change from 0 to 204800000 [ 4044.022259] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4045.388629] loop0: detected capacity change from 0 to 204800000 [ 4045.400236] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4046.745470] loop0: detected capacity change from 0 to 204800000 [ 4046.754126] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4048.070055] loop0: detected capacity change from 0 to 204800000 [ 4048.101426] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4048.111477] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4048.145884] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4048.149404] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4048.153343] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4048.156769] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4048.160268] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4048.163717] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4048.167184] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4048.170748] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4048.174496] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4048.178376] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4048.182347] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4048.185804] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4048.189361] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4048.193078] Lustre: 267440:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4050.814973] LustreError: 267703:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4050.858807] Lustre: server umount MGS complete [ 4051.161793] loop0: detected capacity change from 0 to 204800000 [ 4051.197282] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4051.378893] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4051.394299] Lustre: lustre-MDT0000: Aborting client recovery [ 4051.716380] loop1: detected capacity change from 0 to 204800000 [ 4051.756398] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4051.793978] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4051.797713] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4051.802127] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4051.805559] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4051.808954] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4051.812450] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4051.816545] Lustre: 267869:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4051.824516] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4051.901529] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4051.930934] Lustre: lustre-MDT0001: Aborting client recovery [ 4052.242698] loop2: detected capacity change from 0 to 204800000 [ 4052.279528] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4052.310336] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4052.314060] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4052.317625] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4052.321211] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4052.325442] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4052.329893] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4052.334540] Lustre: 267950:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4052.386934] Lustre: Found index 0 for lustre-OST0000, updating log [ 4052.424618] Lustre: lustre-OST0000: Aborting recovery [ 4052.745337] loop3: detected capacity change from 0 to 204800000 [ 4052.809131] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4052.843173] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4052.846961] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4052.851608] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4052.857246] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4052.861062] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4052.865676] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4052.869557] Lustre: 268034:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4052.905409] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4052.908337] Lustre: Skipped 1 previous similar message [ 4058.089853] LustreError: 267813:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 4062.178564] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4062.179189] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4062.179723] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4062.181692] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4062.192200] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4062.194721] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4063.647583] LustreError: 267781:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 260us: rc = -2 [ 4077.536523] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4077.537177] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4077.537496] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4077.541439] Lustre: Skipped 1 previous similar message [ 4080.959038] LustreError: 268389:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4080.962172] LustreError: 268389:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4081.002768] Lustre: server umount lustre-MDT0000 complete [ 4081.305596] LustreError: 267764:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451163 with bad export cookie 10573342896717874417 [ 4081.307162] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4081.309293] LustreError: 267764:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4082.655575] LustreError: 267976:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4082.655916] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4082.656163] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4082.656170] Lustre: Skipped 3 previous similar messages [ 4082.660674] LustreError: 267976:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 4082.664610] Lustre: Skipped 1 previous similar message [ 4086.687465] LustreError: 267976:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4086.687563] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4086.691964] LustreError: 267976:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 4086.698012] Lustre: Skipped 2 previous similar messages [ 4087.549485] LustreError: 268440:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4087.552319] LustreError: 268440:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4087.612357] Lustre: server umount lustre-MDT0001 complete [ 4093.998046] LustreError: 268491:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4093.999531] LustreError: 268491:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4094.023985] Lustre: server umount lustre-OST0000 complete [ 4100.396102] LustreError: 268543:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4100.399195] LustreError: 268543:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4100.453905] Lustre: server umount lustre-OST0001 complete [ 4103.081241] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:59:44 (1770451184) [ 4111.976762] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 4115.117218] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4127.038456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4127.148079] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4127.161328] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4127.197187] Lustre: lustre-MDT0000: new disk, initializing [ 4127.222099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4127.228558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4128.519887] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4132.644732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4132.684303] Lustre: 272075:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4132.708284] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4132.710037] Lustre: Skipped 1 previous similar message [ 4132.747439] Lustre: lustre-MDT0001: new disk, initializing [ 4132.782867] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4132.796234] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4132.799809] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4134.069637] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4137.117743] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4138.387826] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4140.730378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4140.858284] Lustre: lustre-OST0000: new disk, initializing [ 4140.861117] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4140.899495] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4142.726356] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4144.955759] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4144.960722] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4145.000694] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4145.886799] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4148.271072] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4153.474469] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4157.999130] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4191.201740] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4191.204296] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4191.206890] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4191.208027] Lustre: Skipped 1 previous similar message [ 4195.308134] LustreError: 275592:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4195.310188] LustreError: 275592:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4195.348775] Lustre: server umount lustre-OST0000 complete [ 4199.391846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4199.394894] LustreError: Skipped 1 previous similar message [ 4199.396729] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4199.403170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4199.404963] Lustre: Skipped 1 previous similar message [ 4202.654996] Lustre: server umount lustre-MDT0000 complete [ 4203.930823] LustreError: 272066:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451285 with bad export cookie 10573342896717877133 [ 4203.935662] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4203.936700] LustreError: 272066:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4204.065727] Lustre: server umount lustre-MDT0001 complete [ 4207.018844] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4208.358113] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4210.050938] Lustre: server umount MGS complete [ 4213.825873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4213.992585] LustreError: 277557:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4214.000199] LustreError: 277569:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4214.026629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4215.443582] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4218.322688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4219.745506] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4221.583622] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4222.754478] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4225.009294] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4226.995150] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4228.715864] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4270.564647] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4270.564980] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4270.567615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4270.567621] Lustre: Skipped 2 previous similar messages [ 4270.569274] Lustre: Skipped 2 previous similar messages [ 4272.431129] LustreError: 280230:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4272.433060] LustreError: 280230:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 4272.471392] Lustre: server umount lustre-OST0000 complete [ 4274.655989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4274.658279] LustreError: Skipped 1 previous similar message [ 4279.899742] Lustre: server umount lustre-MDT0000 complete [ 4280.800441] LustreError: 277583:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4280.806067] LustreError: 277583:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 2 previous similar messages [ 4281.205037] LustreError: 277585:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451362 with bad export cookie 10573342896717878946 [ 4281.208262] LustreError: 277585:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4281.208362] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4281.322378] Lustre: server umount lustre-MDT0001 complete [ 4284.905231] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 4285.887939] Key type lgssc unregistered [ 4286.018337] LNet: 281163:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4286.021726] LNetError: 281163:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4286.030332] LNet: Removed LNI 192.168.202.125@tcp [ 4286.312054] Key type .llcrypt unregistered [ 4286.313776] Key type ._llcrypt unregistered [ 4298.195987] Key type ._llcrypt registered [ 4298.197080] Key type .llcrypt registered [ 4298.240731] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 4304.561359] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4304.978093] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4305.002954] alg: No test for adler32 (adler32-zlib) [ 4305.864774] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 4305.954637] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 4307.535108] Key type lgssc registered [ 4307.960341] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4319.615175] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4319.620525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4320.714037] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4320.726357] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4320.762550] Lustre: lustre-MDT0000: new disk, initializing [ 4320.786098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4320.792679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4322.009054] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4326.101355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4326.140271] Lustre: 285054:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4326.164268] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4326.166178] Lustre: Skipped 1 previous similar message [ 4326.202949] Lustre: lustre-MDT0001: new disk, initializing [ 4326.238808] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4326.250454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4326.254427] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4327.497262] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4330.517953] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4331.674527] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4333.956037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4334.060290] Lustre: lustre-OST0000: new disk, initializing [ 4334.062530] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4334.091440] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4335.867685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4335.872235] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4335.906019] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4336.005829] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4339.083648] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4342.458708] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4344.962291] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4349.485178] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4397.537725] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4397.538451] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4397.542668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4397.545799] LustreError: Skipped 1 previous similar message [ 4402.657685] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4402.659945] Lustre: Skipped 2 previous similar messages [ 4407.263107] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4407.284864] LustreError: 288463:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4407.322444] Lustre: server umount lustre-OST0000 complete [ 4412.896539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4412.897302] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4412.900955] Lustre: Skipped 2 previous similar messages [ 4412.903988] Lustre: Skipped 1 previous similar message [ 4414.724382] LustreError: 288665:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4414.727479] LustreError: 288665:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4414.757888] Lustre: server umount lustre-MDT0000 complete [ 4415.993977] LustreError: 287537:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451497 with bad export cookie 17255373246420994582 [ 4415.996583] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4415.996959] LustreError: 287537:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4416.063788] LustreError: 288866:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4416.065434] LustreError: 288866:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4416.119517] Lustre: server umount lustre-MDT0001 complete [ 4418.994530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4420.306364] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4421.879042] LustreError: 289808:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4421.881361] LustreError: 289808:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4421.901513] Lustre: server umount MGS complete [ 4425.721637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4425.907074] LustreError: 290442:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4425.932575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4427.379610] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4430.221367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4430.349923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4431.711568] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4433.665013] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4435.006702] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4437.323383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4439.312850] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4441.156861] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4476.384400] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4476.386811] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4476.390912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4477.413487] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4481.135324] LustreError: 293107:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4481.137430] LustreError: 293107:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4481.173344] Lustre: server umount lustre-OST0000 complete [ 4482.530038] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4482.530942] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4482.536987] Lustre: Skipped 2 previous similar messages [ 4482.539436] Lustre: Skipped 2 previous similar messages [ 4488.687302] Lustre: server umount lustre-MDT0000 complete [ 4490.276315] LustreError: 292953:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451572 with bad export cookie 17255373246420996395 [ 4490.279791] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4490.280149] LustreError: 292953:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4490.351146] LustreError: 293510:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4490.353020] LustreError: 293510:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4490.403931] Lustre: server umount lustre-MDT0001 complete [ 4494.367894] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 4495.395490] Key type lgssc unregistered [ 4495.526417] LNet: 294041:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4495.530792] LNetError: 294041:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4495.541310] LNet: Removed LNI 192.168.202.125@tcp [ 4495.945827] Key type .llcrypt unregistered [ 4495.947643] Key type ._llcrypt unregistered [ 4502.561276] Key type ._llcrypt registered [ 4502.562300] Key type .llcrypt registered [ 4502.596826] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 4508.775973] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4509.123642] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4509.133600] alg: No test for adler32 (adler32-zlib) [ 4509.991001] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 4510.070615] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 4511.647217] Key type lgssc registered [ 4512.082372] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4520.169074] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4525.122491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4525.129452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4526.218992] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4526.231197] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4526.268735] Lustre: lustre-MDT0000: new disk, initializing [ 4526.292047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4526.298316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4527.544334] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4531.636060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4531.741081] Lustre: lustre-OST0000: new disk, initializing [ 4531.744122] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4531.746525] Lustre: Skipped 1 previous similar message [ 4531.770338] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4532.623235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4532.628258] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4532.639899] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4533.710255] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4706.785410] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4706.793514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4708.416658] LustreError: 298948:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 4708.519593] Lustre: server umount lustre-MDT0000 complete [ 4709.804609] LustreError: 297205:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451791 with bad export cookie 15368414402693785666 [ 4709.808025] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4709.808798] LustreError: 297205:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4709.838473] LustreError: 299150:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4709.841786] LustreError: 299150:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4709.895383] Lustre: server umount lustre-OST0000 complete [ 4718.855174] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 4722.524191] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4735.738084] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4735.837640] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4735.849738] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4735.891625] Lustre: lustre-MDT0000: new disk, initializing [ 4735.914981] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4735.921804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4737.277184] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4742.038051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4742.078179] Lustre: 302176:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4742.106710] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4742.108424] Lustre: Skipped 1 previous similar message [ 4742.146634] Lustre: lustre-MDT0001: new disk, initializing [ 4742.184486] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4742.199197] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4742.205371] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4743.571076] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4746.695274] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4747.986692] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4750.263810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4750.358242] Lustre: lustre-OST0000: new disk, initializing [ 4750.360131] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4750.387269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4752.389450] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4755.498652] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4756.817544] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4756.985289] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4756.987867] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4757.021951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4757.897085] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 4759.155315] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4759.228684] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4762.082367] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4762.082545] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4762.086929] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4762.088072] LustreError: Skipped 1 previous similar message [ 4765.934112] LustreError: 305113:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4765.935907] LustreError: 305113:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4765.976397] Lustre: server umount lustre-OST0000 complete [ 4767.711894] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4767.714183] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4767.717320] Lustre: Skipped 1 previous similar message [ 4767.718848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4767.720522] Lustre: Skipped 1 previous similar message [ 4772.320168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4772.320527] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4773.379292] LustreError: 305314:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4773.382145] LustreError: 305314:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4773.412392] Lustre: server umount lustre-MDT0000 complete [ 4774.698248] LustreError: 302167:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451856 with bad export cookie 15368414402696726443 [ 4774.701884] LustreError: 302167:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4774.702545] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4774.826316] Lustre: server umount lustre-MDT0001 complete [ 4775.925746] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4776.506152] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:10:57 (1770451857) [ 4778.690176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4778.871146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4780.206760] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4783.062544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4784.474376] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4786.267292] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4787.495358] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4789.907780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4791.978570] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4793.911979] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4796.294415] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4796.364798] Lustre: lustre-OST0001: new disk, initializing [ 4796.366783] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4796.401394] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4796.403356] Lustre: Skipped 2 previous similar messages [ 4798.379182] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4805.134386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4805.140348] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4805.161164] LustreError: 309138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4806.178653] LustreError: 309143:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4806.182585] LustreError: 309143:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4807.202445] LustreError: 309143:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4807.205845] LustreError: 309143:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4807.423105] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4807.503738] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4808.705214] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa004033c1800.ost_server_uuid 50 [ 4809.250923] LustreError: 309138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4809.253960] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa004033c1800.ost_server_uuid in FULL state after 0 sec [ 4809.254741] LustreError: 309138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4810.495687] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa004033c1800.ost_server_uuid 50 [ 4811.028440] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa004033c1800.ost_server_uuid in FULL state after 0 sec [ 4812.330166] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4825.572874] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4825.574758] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4825.578855] Lustre: Skipped 1 previous similar message [ 4825.582360] Lustre: Skipped 1 previous similar message [ 4835.811957] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4835.814117] Lustre: Skipped 6 previous similar messages [ 4837.343090] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4837.363173] LustreError: 309888:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 4837.365397] LustreError: 309888:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 4837.385426] Lustre: server umount lustre-OST0001 complete [ 4839.392553] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4839.394961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4840.931874] LustreError: 307679:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4840.938641] LustreError: 307679:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 4845.132233] Lustre: server umount lustre-OST0000 complete [ 4849.631930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4849.635155] LustreError: Skipped 1 previous similar message [ 4852.545167] LustreError: 310292:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4852.547578] LustreError: 310292:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4852.582489] Lustre: server umount lustre-MDT0000 complete [ 4853.906790] LustreError: 306068:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770451935 with bad export cookie 15368414402696727311 [ 4853.909951] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4853.912849] LustreError: 306068:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4854.046760] Lustre: server umount lustre-MDT0001 complete [ 4857.794740] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 4858.904737] Key type lgssc unregistered [ 4859.033576] LNet: 311024:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4859.036418] LNetError: 311024:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4859.046355] LNet: Removed LNI 192.168.202.125@tcp [ 4859.395889] Key type .llcrypt unregistered [ 4859.397328] Key type ._llcrypt unregistered [ 4868.883646] Key type ._llcrypt registered [ 4868.884803] Key type .llcrypt registered [ 4868.929577] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 4869.395486] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4869.445597] alg: No test for adler32 (adler32-zlib) [ 4870.315488] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 4870.412479] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 4871.999186] Key type lgssc registered [ 4872.379933] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4874.957087] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4874.962734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4876.124578] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4877.393131] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4879.284247] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4881.651783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4881.777143] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4883.164298] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4886.217059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4886.333314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4888.160174] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4889.890577] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4891.492150] LustreError: 313952:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4891.498928] LustreError: 313952:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 2 previous similar messages [ 4892.195769] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4892.270073] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4894.118254] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 4895.946715] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4900.845364] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4900.846111] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4902.677136] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4902.747690] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4979.679835] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4979.682497] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4979.686622] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4982.753558] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4982.755850] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4982.756403] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4982.760892] Lustre: Skipped 1 previous similar message [ 4983.980096] LustreError: 315816:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4983.998741] Lustre: server umount lustre-OST0001 complete [ 4987.874754] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4987.875982] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4987.879589] LustreError: 315395:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4987.881636] Lustre: Skipped 1 previous similar message [ 4987.884161] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4987.886088] LustreError: 315395:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 4991.597099] LustreError: 316017:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4991.600438] LustreError: 316017:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4991.636423] Lustre: server umount lustre-OST0000 complete [ 4992.992576] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4992.997740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4993.000030] Lustre: Skipped 2 previous similar messages [ 4994.944911] LustreError: 316218:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4994.947011] LustreError: 316218:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4994.977255] Lustre: server umount lustre-MDT0000 complete [ 4996.207512] LustreError: 313220:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770452078 with bad export cookie 10311170347606755163 [ 4996.211654] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4996.213182] LustreError: 313220:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4996.342412] Lustre: server umount lustre-MDT0001 complete [ 4999.865127] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 5000.841310] Key type lgssc unregistered [ 5000.978394] LNet: 316949:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5000.980840] LNetError: 316949:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5000.988302] LNet: Removed LNI 192.168.202.125@tcp [ 5001.293176] Key type .llcrypt unregistered [ 5001.294773] Key type ._llcrypt unregistered [ 5019.356229] Key type ._llcrypt registered [ 5019.357429] Key type .llcrypt registered [ 5019.400296] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 5019.785940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5019.812126] alg: No test for adler32 (adler32-zlib) [ 5020.676581] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 5020.780871] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 5022.367138] Key type lgssc registered [ 5022.731637] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5025.744884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5025.750579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5026.917839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5028.257968] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5031.025058] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5031.152744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5032.444974] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5033.410499] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5035.591357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5035.720889] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5036.776291] LustreError: 320646:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5036.779837] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5036.782423] LustreError: 320646:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 2 previous similar messages [ 5037.598762] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5040.418960] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5040.492546] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5041.467167] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5042.353716] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5046.499345] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5052.154890] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5062.111930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5062.114386] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5062.118486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5063.139502] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5063.141225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5063.143394] Lustre: Skipped 2 previous similar messages [ 5063.146331] Lustre: Skipped 2 previous similar messages [ 5066.820648] LustreError: 322592:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5066.861847] Lustre: server umount lustre-MDT0000 complete [ 5068.066470] LustreError: 320295:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770452149 with bad export cookie 2818295925161723832 [ 5068.070270] LustreError: 320295:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5068.070876] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5068.142815] LustreError: 322794:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5068.144542] LustreError: 322794:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5068.190514] Lustre: server umount lustre-MDT0001 complete [ 5079.668085] LustreError: 322995:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5079.670212] LustreError: 322995:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5079.685750] Lustre: server umount lustre-OST0000 complete [ 5091.123383] LustreError: 323196:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5091.125666] LustreError: 323196:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5091.169281] Lustre: server umount lustre-OST0001 complete [ 5096.501719] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 5099.950734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5100.093026] LustreError: 324370:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5100.113420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5101.414834] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5104.255154] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5105.737071] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5106.760643] Lustre: 325477:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5109.123699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5109.278447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5109.281507] Lustre: Skipped 1 previous similar message [ 5110.307428] LustreError: 325829:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5110.313583] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5111.321409] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5114.208062] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5115.307724] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5116.240379] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5119.500835] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5120.750412] Lustre: 327316:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5130.207812] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5130.210034] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5130.213955] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5130.215793] Lustre: Skipped 1 previous similar message [ 5132.867061] LustreError: 327421:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5132.869504] LustreError: 327421:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5132.899956] Lustre: server umount lustre-MDT0000 complete [ 5134.145228] LustreError: 324352:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770452215 with bad export cookie 2818295925161725491 [ 5134.150540] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5134.150802] LustreError: 324352:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5134.277588] Lustre: server umount lustre-MDT0001 complete [ 5146.036099] LustreError: 327823:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5146.038958] LustreError: 327823:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5146.058062] Lustre: server umount lustre-OST0000 complete [ 5157.659206] Lustre: server umount lustre-OST0001 complete [ 5158.932196] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5159.508508] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5160.124769] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:17:21 (1770452241) [ 5167.887367] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 5171.333198] loop0: detected capacity change from 0 to 1099511627776 [ 5171.497057] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5171.499801] *** VALIDATE xfs *** [ 5171.505732] XFS (loop0): Mounting V5 Filesystem [ 5171.516873] XFS (loop0): Ending clean mount [ 5171.913958] loop1: detected capacity change from 0 to 18691697672192 [ 5191.961418] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 5195.391453] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 5207.308885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5207.452895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5207.471966] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5207.530334] Lustre: lustre-MDT0000: new disk, initializing [ 5207.560164] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5207.562063] Lustre: Skipped 1 previous similar message [ 5207.567749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5208.782801] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5212.902414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5212.942329] Lustre: 332382:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5212.965857] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5212.967607] Lustre: Skipped 1 previous similar message [ 5213.003639] Lustre: lustre-MDT0001: new disk, initializing [ 5213.050716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5213.054316] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5214.316316] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5217.499573] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5218.923133] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5221.597128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5221.700756] Lustre: lustre-OST0000: new disk, initializing [ 5221.702674] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5223.620528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5223.623518] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5223.663503] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5223.785047] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5227.572452] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5229.340764] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5229.414730] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5231.040870] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5231.158889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5234.144854] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5234.146029] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5234.147504] LustreError: Skipped 1 previous similar message [ 5234.153706] Lustre: Skipped 3 previous similar messages [ 5234.156028] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5234.159474] Lustre: Skipped 4 previous similar messages [ 5237.934204] LustreError: 335311:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5237.936193] LustreError: 335311:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5237.974708] Lustre: server umount lustre-OST0000 complete [ 5239.264859] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5239.265686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5239.269145] Lustre: Skipped 1 previous similar message [ 5239.274649] Lustre: Skipped 1 previous similar message [ 5245.351889] Lustre: server umount lustre-MDT0000 complete [ 5246.618588] LustreError: 332372:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770452328 with bad export cookie 2818295925161727045 [ 5246.622896] LustreError: 332372:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5246.622920] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5246.761379] Lustre: server umount lustre-MDT0001 complete [ 5249.367134] XFS (loop0): Unmounting Filesystem [ 5250.903355] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:18:51 (1770452331) [ 5253.006635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5253.169670] LustreError: 336529:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5253.196584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5253.198493] Lustre: Skipped 2 previous similar messages [ 5254.554909] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5257.435374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5258.878144] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5260.581254] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5261.810528] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5263.952317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5265.935909] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5267.697294] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5282.788531] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5282.788756] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5282.790201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5282.790208] Lustre: Skipped 2 previous similar messages [ 5282.793515] Lustre: Skipped 1 previous similar message [ 5283.246308] LustreError: 338888:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5283.248675] LustreError: 338888:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5283.286260] Lustre: server umount lustre-OST0000 complete [ 5286.742920] Lustre: server umount lustre-MDT0000 complete [ 5287.906106] LustreError: 338682:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5287.910792] LustreError: 338682:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 5287.960396] LustreError: 336509:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770452369 with bad export cookie 2818295925161727927 [ 5287.963694] LustreError: 336509:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5287.964891] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5288.109278] Lustre: server umount lustre-MDT0001 complete [ 5291.576965] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 5292.530869] Key type lgssc unregistered [ 5292.667432] LNet: 339821:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5292.670074] LNetError: 339821:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5292.679364] LNet: Removed LNI 192.168.202.125@tcp [ 5292.977090] Key type .llcrypt unregistered [ 5292.978199] Key type ._llcrypt unregistered [ 5303.184193] Key type ._llcrypt registered [ 5303.185183] Key type .llcrypt registered [ 5303.227162] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 5303.564862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5303.621631] alg: No test for adler32 (adler32-zlib) [ 5304.473564] Lustre: Lustre: Build Version: 2.17.50_44_g630b48b [ 5304.550621] LNet: Added LNI 192.168.202.125@tcp [8/256/0/180] [ 5306.135131] Key type lgssc registered [ 5306.511513] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5309.066489] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5309.072810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5310.221800] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5311.497283] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5314.427187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5314.583044] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5315.973187] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5317.778087] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5319.055474] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5321.655201] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5321.806604] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5323.665395] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5325.578822] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5345.248404] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5345.253562] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5345.262169] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5346.239952] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.25@tcp (stopping) [ 5346.792389] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5350.088059] LustreError: 343655:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5350.121281] Lustre: server umount lustre-MDT0001 complete [ 5351.347247] LustreError: 342652:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.25@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5351.352354] LustreError: 342652:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 2 previous similar messages [ 5351.907663] LustreError: 341344:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5351.911527] LustreError: 341344:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 5354.685187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5354.737947] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5354.741816] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5354.766813] Lustre: 341333:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5354.980734] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5356.552510] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5360.108762] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5363.691942] LustreError: 344795:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5363.699033] LustreError: 344795:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5363.705132] LustreError: 341335:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5363.742670] LustreError: 344802:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.125@tcp: cfg command failed: rc = -17 [ 5363.750537] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5363.750537] [ 5658.552581] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.25@tcp (stopping) [ 5658.555075] Lustre: Skipped 2 previous similar messages [ 5662.179064] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5662.179541] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5662.188231] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5662.191513] Lustre: Skipped 1 previous similar message [ 5664.316642] LustreError: 355915:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5664.318562] LustreError: 355915:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5664.342129] Lustre: server umount lustre-MDT0001 complete [ 5667.298106] LustreError: 344675:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5667.305219] LustreError: 344675:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 5667.996378] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5668.051143] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5668.055488] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5668.074618] Lustre: 341332:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5668.245902] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5669.556121] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5673.445027] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5674.471162] LustreError: 356982:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5674.473559] LustreError: 356982:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5674.476187] LustreError: 341335:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5674.478573] LustreError: 341335:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5674.495070] LustreError: 356985:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 5674.497365] LustreError: 356985:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [ 5674.499247] LustreError: 356985:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.125@tcp: cfg command failed: rc = -17 [ 5674.502137] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.125@tcp [ 5674.502137] [ 5973.478146] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5973.486551] Lustre: Skipped 1 previous similar message [ 5973.490468] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5973.495026] Lustre: Skipped 2 previous similar messages [ 5983.714940] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5983.719895] Lustre: Skipped 5 previous similar messages [ 5986.271191] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5986.338196] LustreError: 368324:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5986.342247] LustreError: 368324:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5986.377316] Lustre: server umount lustre-MDT0001 complete [ 5988.832432] LustreError: 341343:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5988.837581] LustreError: 341343:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 1 previous similar message [ 5990.649405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5990.708291] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5990.710656] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5990.727447] Lustre: 341330:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5990.867934] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5992.422459] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 5996.010797] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5998.570674] LustreError: 369443:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5998.575436] LustreError: 369443:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5998.580908] LustreError: 341335:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5998.585845] LustreError: 341335:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5998.618781] LustreError: 369446:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5998.629309] LustreError: 369446:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.125@tcp: cfg command failed: rc = -114 [ 5998.635550] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5998.635550] [ 6308.321881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6308.326860] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6308.334368] Lustre: Skipped 1 previous similar message [ 6308.337839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6308.341706] Lustre: Skipped 1 previous similar message [ 6312.720566] LustreError: 380098:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6312.724346] LustreError: 380098:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6312.761725] Lustre: server umount lustre-MDT0000 complete [ 6314.207983] LustreError: 342957:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770453396 with bad export cookie 739042217344048927 [ 6314.214161] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6314.215882] LustreError: 342957:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6316.512586] LustreError: 344675:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6316.512962] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6316.529204] Lustre: Skipped 2 previous similar messages [ 6320.386467] LustreError: 380299:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6320.388437] LustreError: 380299:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6320.435619] Lustre: server umount lustre-MDT0001 complete [ 6321.838261] Lustre: server umount lustre-OST0000 complete [ 6325.648937] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 6329.999958] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 6346.924204] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:37:07 (1770453427) [ 6350.411944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6350.558182] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6350.579815] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6350.635645] Lustre: lustre-MDT0000: new disk, initializing [ 6350.674416] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6350.685143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6352.451324] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 6357.939729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6358.014994] Lustre: 383019:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6358.051768] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6358.054708] Lustre: Skipped 1 previous similar message [ 6358.106629] Lustre: lustre-MDT0001: new disk, initializing [ 6358.148722] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6358.165952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6358.171265] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6359.769683] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 6363.638334] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6365.309727] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6368.783745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6368.963642] Lustre: lustre-OST0000: new disk, initializing [ 6368.967555] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6369.027183] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6370.510645] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6370.517454] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6370.574851] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6371.794427] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_default_debug -1 all [ 6375.672283] Lustre: DEBUG MARKER: oleg225-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6378.976387] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6378.981568] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6378.990770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6378.994872] Lustre: Skipped 5 previous similar messages [ 6385.041734] LustreError: 385458:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6385.045676] LustreError: 385458:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6385.094011] Lustre: server umount lustre-MDT0000 complete [ 6386.148876] LustreError: 383026:0:(ldlm_lib.c:1185:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6386.159752] LustreError: 383026:0:(ldlm_lib.c:1185:target_handle_connect()) Skipped 3 previous similar messages [ 6386.875172] LustreError: 383011:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1770453468 with bad export cookie 739042217344050887 [ 6386.881958] LustreError: 383811:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6386.884768] LustreError: MGC192.168.202.125@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6386.886169] LustreError: 383011:0:(ldlm_lockd.c:2597:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6387.052114] Lustre: server umount lustre-MDT0001 complete [ 6399.292886] Lustre: server umount lustre-OST0000 complete [ 6412.383409] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing set_hostid [ 6416.405823] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing load_modules_local [ 6429.130431] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6310 sec ========== 03:38:29 (1770453509) [ 6429.698603] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:38:30 (1770453510) === [ 6431.126125] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:38:32 (1770453512) === [ 6447.395268] Lustre: DEBUG MARKER: oleg225-server.virtnet: executing unload_modules_local [ 6448.904202] Key type lgssc unregistered [ 6449.092710] LNet: 390955:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6449.096550] LNetError: 390955:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6449.106636] LNet: Removed LNI 192.168.202.125@tcp [ 6449.633500] Key type .llcrypt unregistered [ 6449.637046] Key type ._llcrypt unregistered