[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 644989212 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003466] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005029] kvm-guest: setup PV IPIs [ 0.009000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009031] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011216] LSM: Security Framework initializing [ 0.012054] Yama: becoming mindful. [ 0.013051] SELinux: Initializing. [ 0.015149] *** VALIDATE selinux *** [ 0.027902] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.035140] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.036196] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.037119] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.038172] *** VALIDATE tmpfs *** [ 0.039544] *** VALIDATE proc *** [ 0.040301] *** VALIDATE cgroup *** [ 0.041009] *** VALIDATE cgroup2 *** [ 0.042280] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.043160] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.045004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.046031] Spectre V2 : User space: Vulnerable [ 0.047009] Speculative Store Bypass: Vulnerable [ 0.049743] debug: unmapping init [mem 0xffffffff97859000-0xffffffff97860fff] [ 0.052303] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.053772] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.054025] ... version: 2 [ 0.055013] ... bit width: 48 [ 0.056010] ... generic registers: 4 [ 0.057011] ... value mask: 0000ffffffffffff [ 0.058014] ... max period: 00007fffffffffff [ 0.059011] ... fixed-purpose events: 3 [ 0.060010] ... event mask: 000000070000000f [ 0.061408] rcu: Hierarchical SRCU implementation. [ 0.063814] smp: Bringing up secondary CPUs ... [ 0.064803] x86: Booting SMP configuration: [ 0.065019] .... node #0, CPUs: #1 #2 #3 [ 0.070016] smp: Brought up 1 node, 4 CPUs [ 0.072016] smpboot: Max logical packages: 1 [ 0.073015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.107691] node 0 deferred pages initialised in 32ms [ 0.110622] devtmpfs: initialized [ 0.111183] x86/mm: Memory block size: 128MB [ 0.113808] gcov: version magic: 0x41383552 [ 0.115385] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.116078] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.117341] pinctrl core: initialized pinctrl subsystem [ 0.118243] [ 0.118618] ************************************************************* [ 0.119019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.120014] ** ** [ 0.121015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.122009] ** ** [ 0.123012] ** This means that this kernel is built to expose internal ** [ 0.124019] ** IOMMU data structures, which may compromise security on ** [ 0.125018] ** your system. ** [ 0.126016] ** ** [ 0.127012] ** If you see this message and you are not debugging the ** [ 0.128017] ** kernel, report this immediately to your vendor! ** [ 0.129017] ** ** [ 0.130014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.131018] ************************************************************* [ 0.133159] NET: Registered protocol family 16 [ 0.134753] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.135183] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.136054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.137687] cpuidle: using governor menu [ 0.140027] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.142750] PCI: Using configuration type 1 for base access [ 0.145123] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.154384] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.156146] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.158063] cryptd: max_cpu_qlen set to 1000 [ 0.161200] ACPI: Added _OSI(Module Device) [ 0.163017] ACPI: Added _OSI(Processor Device) [ 0.165016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.167012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.172437] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.179914] ACPI: Interpreter enabled [ 0.180062] ACPI: PM: (supports S0 S3 S4 S5) [ 0.181011] ACPI: Using IOAPIC for interrupt routing [ 0.182099] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.183535] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.197211] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.198037] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.199018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.200188] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.206445] acpiphp: Slot [2] registered [ 0.208238] acpiphp: Slot [5] registered [ 0.210136] acpiphp: Slot [6] registered [ 0.212119] acpiphp: Slot [7] registered [ 0.214114] acpiphp: Slot [8] registered [ 0.216240] acpiphp: Slot [9] registered [ 0.218182] acpiphp: Slot [10] registered [ 0.220116] acpiphp: Slot [3] registered [ 0.222161] acpiphp: Slot [4] registered [ 0.224162] acpiphp: Slot [11] registered [ 0.225206] acpiphp: Slot [12] registered [ 0.227208] acpiphp: Slot [13] registered [ 0.229122] acpiphp: Slot [14] registered [ 0.231099] acpiphp: Slot [15] registered [ 0.232090] acpiphp: Slot [16] registered [ 0.233062] acpiphp: Slot [17] registered [ 0.235088] acpiphp: Slot [18] registered [ 0.236083] acpiphp: Slot [19] registered [ 0.237096] acpiphp: Slot [20] registered [ 0.239070] acpiphp: Slot [21] registered [ 0.241091] acpiphp: Slot [22] registered [ 0.242108] acpiphp: Slot [23] registered [ 0.244105] acpiphp: Slot [24] registered [ 0.246227] acpiphp: Slot [25] registered [ 0.247209] acpiphp: Slot [26] registered [ 0.249088] acpiphp: Slot [27] registered [ 0.250087] acpiphp: Slot [28] registered [ 0.251096] acpiphp: Slot [29] registered [ 0.252120] acpiphp: Slot [30] registered [ 0.254104] acpiphp: Slot [31] registered [ 0.256076] PCI host bridge to bus 0000:00 [ 0.257016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.260022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.263042] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.268036] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.271042] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.275052] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.277227] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.281521] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.287000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.305029] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.310109] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.313028] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.315015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.318015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.320343] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.322765] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.325042] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.327841] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.333013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.348014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.355015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.362032] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.369017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.380027] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.402017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.420089] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.431020] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.442016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.466013] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.482158] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.489016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.496019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.514022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.529311] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.540016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.546018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.566018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.583029] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.590017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.598016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.619017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.629853] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.638029] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.643014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.665056] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.678571] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.681501] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.684830] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.691000] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.693239] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.698160] iommu: Default domain type: Passthrough [ 0.700245] SCSI subsystem initialized [ 0.702211] ACPI: bus type USB registered [ 0.704098] usbcore: registered new interface driver usbfs [ 0.706121] usbcore: registered new interface driver hub [ 0.709075] usbcore: registered new device driver usb [ 0.711156] pps_core: LinuxPPS API ver. 1 registered [ 0.713010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.716063] PTP clock support registered [ 0.718123] EDAC MC: Ver: 3.0.0 [ 0.720129] PCI: Using ACPI for IRQ routing [ 0.723456] NetLabel: Initializing [ 0.724009] NetLabel: domain hash size = 128 [ 0.726009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.728076] NetLabel: unlabeled traffic allowed by default [ 0.730162] vgaarb: loaded [ 0.732264] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.734013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.741940] clocksource: Switched to clocksource kvm-clock [ 0.852649] VFS: Disk quotas dquot_6.6.0 [ 0.854369] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.857103] *** VALIDATE ramfs *** [ 0.858491] *** VALIDATE hugetlbfs *** [ 0.860176] pnp: PnP ACPI init [ 0.862625] pnp: PnP ACPI: found 6 devices [ 0.877846] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.881445] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.883754] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.886062] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.888614] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.891204] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.894215] NET: Registered protocol family 2 [ 0.896610] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.901692] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.905538] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.910828] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.914524] TCP: Hash tables configured (established 65536 bind 65536) [ 0.917514] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.920648] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.923576] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.926581] NET: Registered protocol family 1 [ 0.928794] RPC: Registered named UNIX socket transport module. [ 0.931022] RPC: Registered udp transport module. [ 0.932819] RPC: Registered tcp transport module. [ 0.934653] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.937162] NET: Registered protocol family 44 [ 0.938842] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.941082] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.943143] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.945679] PCI: CLS 0 bytes, default 64 [ 0.947403] Unpacking initramfs... [ 2.405410] debug: unmapping init [mem 0xffff9eafbcc54000-0xffff9eafbffbffff] [ 2.409285] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.411741] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.414803] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.937925] Initialise system trusted keyrings [ 2.939582] Key type blacklist registered [ 2.941401] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.950838] zbud: loaded [ 2.954341] *** VALIDATE nfs *** [ 2.955504] *** VALIDATE nfs4 *** [ 2.957201] pstore: using deflate compression [ 2.962758] Platform Keyring initialized [ 3.081954] NET: Registered protocol family 38 [ 3.083670] Key type asymmetric registered [ 3.085525] Asymmetric key parser 'x509' registered [ 3.087754] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.091755] io scheduler mq-deadline registered [ 3.093759] io scheduler kyber registered [ 3.095237] io scheduler bfq registered [ 3.097244] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.100410] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.103614] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.106767] ACPI: Power Button [PWRF] [ 3.113221] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.120339] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.145757] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.157113] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.182844] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.211293] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.237864] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.242126] Non-volatile memory driver v1.3 [ 3.243698] Linux agpgart interface v0.103 [ 3.274711] virtio_blk virtio1: [vda] 145176 512-byte logical blocks (74.3 MB/70.9 MiB) [ 3.277658] vda: detected capacity change from 0 to 74330112 [ 3.290858] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.293822] vdb: detected capacity change from 0 to 1073741824 [ 3.315241] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.318587] vdc: detected capacity change from 0 to 2621440000 [ 3.371756] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.375132] vdd: detected capacity change from 0 to 2621440000 [ 3.404637] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.408535] vde: detected capacity change from 0 to 4294967296 [ 3.428804] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.431083] vdf: detected capacity change from 0 to 4294967296 [ 3.444106] libphy: Fixed MDIO Bus: probed [ 3.450751] usbcore: registered new interface driver usbserial_generic [ 3.456616] usbserial: USB Serial support registered for generic [ 3.460353] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.465627] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.468122] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.470564] mousedev: PS/2 mouse device common for all mice [ 3.474193] rtc_cmos 00:05: RTC can wake from S4 [ 3.477631] rtc_cmos 00:05: registered as rtc0 [ 3.479738] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.482355] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.482409] intel_pstate: CPU model not supported [ 3.483898] hid: raw HID events driver (C) Jiri Kosina [ 3.498831] usbcore: registered new interface driver usbhid [ 3.500853] usbhid: USB HID core driver [ 3.502513] drop_monitor: Initializing network drop monitor service [ 3.505258] Initializing XFRM netlink socket [ 3.507463] NET: Registered protocol family 10 [ 3.509286] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.515232] Segment Routing with IPv6 [ 3.516430] NET: Registered protocol family 17 [ 3.518640] mpls_gso: MPLS GSO support [ 3.521481] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.529236] RAS: Correctable Errors collector initialized. [ 3.531459] AVX version of gcm_enc/dec engaged. [ 3.533206] AES CTR mode by8 optimization enabled [ 3.619411] sched_clock: Marking stable (3619377986, 0)->(4655957799, -1036579813) [ 3.628334] registered taskstats version 1 [ 3.631369] Loading compiled-in X.509 certificates [ 3.633988] zswap: loaded using pool lzo/zbud [ 3.674614] Key type big_key registered [ 3.687950] Key type encrypted registered [ 3.689678] ima: No TPM chip found, activating TPM-bypass! [ 3.691892] ima: Allocated hash algorithm: sha1 [ 3.693538] ima: No architecture policies found [ 3.695270] evm: Initialising EVM extended attributes: [ 3.697539] evm: security.selinux [ 3.698739] evm: security.ima [ 3.699892] evm: security.capability [ 3.701410] evm: HMAC attrs: 0x1 [ 3.704446] rtc_cmos 00:05: setting system clock to 2026-07-07 16:16:52 UTC (1783441012) [ 3.710744] debug: unmapping init [mem 0xffffffff98803000-0xffffffff989fffff] [ 3.713854] debug: unmapping init [mem 0xffffffff97582000-0xffffffff97858fff] [ 3.721230] Write protecting the kernel read-only data: 28672k [ 3.727051] debug: unmapping init [mem 0xffffffff95c03000-0xffffffff95dfffff] [ 3.730116] debug: unmapping init [mem 0xffffffff96514000-0xffffffff965fffff] [ 3.772326] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.781428] systemd[1]: Detected virtualization kvm. [ 3.783628] systemd[1]: Detected architecture x86-64. [ 3.785724] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.809691] systemd[1]: No hostname configured. [ 3.811490] systemd[1]: Set hostname to . [ 3.813481] random: systemd: uninitialized urandom read (16 bytes read) [ 3.816329] systemd[1]: Initializing machine ID from random generator. [ 3.960080] random: ln: uninitialized urandom read (6 bytes read) [ 4.097709] random: systemd: uninitialized urandom read (16 bytes read) [ 4.101276] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.111648] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.117793] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.387544] device-mapper: uevent: version 1.0.3 [ 5.390197] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 6.762544] random: fast init done [ 6.808660] virtio_net virtio0 ens2: renamed from eth0 [ 7.469101] scsi host0: ata_piix [ 7.496169] scsi host1: ata_piix [ 7.497766] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.500313] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 12.395033] random: crng init done [ 12.472112] random: 7 urandom warning(s) missed due to ratelimiting [ 12.915961] dracut-initqueue[597]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 13.999246] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.845960] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.276725] SELinux: Disabled at runtime. [ 16.350432] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.362650] systemd[1]: Detected virtualization kvm. [ 16.365223] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.225352] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.230262] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.238291] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.243407] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.247100] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.254423] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.266721] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Kernel Debug File System... Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Paths. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ 17.550434] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting POSIX Message Queue File System... Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-getty.slice. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 18.384905] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 19.581239] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 19.619322] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 20.035904] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 20.146073] EDAC sbridge: Ver: 1.1.2 [ 22.250849] Key type dns_resolver registered [ 22.670538] NFS: Registering the id_resolver key type [ 22.673382] Key type id_resolver registered [ 22.676189] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started Login Service. [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg242-server login: [ 50.300114] hrtimer: interrupt took 7104138 ns [ 103.364981] libcfs: loading out-of-tree module taints kernel. [ 103.417840] Key type ._llcrypt registered [ 103.421400] Key type .llcrypt registered [ 103.592578] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 124.806537] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 126.288541] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 126.308978] alg: No test for adler32 (adler32-zlib) [ 127.687904] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 128.505192] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 130.321244] Key type lgssc registered [ 131.866569] Lustre: Echo OBD driver; http://www.lustre.org/ [ 154.136614] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 198.048825] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 212.473300] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 212.502375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 213.794180] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 213.829580] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 213.945331] Lustre: lustre-MDT0000: new disk, initializing [ 214.049842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 214.082477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 218.896946] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 234.055546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 234.149510] Lustre: 6526:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 234.189464] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 234.192397] Lustre: Skipped 1 previous similar message [ 234.282209] Lustre: lustre-MDT0001: new disk, initializing [ 234.329799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 234.350691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 234.365229] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 239.362992] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.882468] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 255.378985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 255.704961] Lustre: lustre-OST0000: new disk, initializing [ 255.713151] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 255.717902] Lustre: 8465:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 255.781136] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 256.022968] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 256.042243] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 256.219625] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 263.097379] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 280.152939] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 280.321043] Lustre: lustre-OST0001: new disk, initializing [ 280.327370] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 280.332269] Lustre: 9542:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 280.400265] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 287.328536] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 287.791194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 287.803113] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 287.893535] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 299.990857] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 306.018241] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 312.244879] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing check_logdir /tmp/testlogs/ [ 318.098725] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing yml_node [ 325.432350] Lustre: DEBUG MARKER: Client: 2.17.54.105 [ 330.285926] Lustre: DEBUG MARKER: MDS: 2.17.54.105 [ 334.113952] Lustre: DEBUG MARKER: OSS: 2.17.54.105 [ 337.221812] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Jul 7 12:22:23 EDT 2026 [ 356.174541] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 357.700940] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 374.752859] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.759569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.765616] Lustre: Skipped 3 previous similar messages [ 374.781324] Lustre: Skipped 3 previous similar messages [ 379.895415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 379.901239] Lustre: Skipped 3 previous similar messages [ 381.130648] Lustre: server umount lustre-MDT0000 complete [ 385.066743] LustreError: 7475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783441393 with bad export cookie 4772611864199670278 [ 385.067927] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 385.080906] LustreError: 7475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 390.111911] LustreError: 9549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 390.124021] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 390.130311] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 390.133372] LustreError: 9549:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 390.158403] Lustre: Skipped 1 previous similar message [ 391.485577] Lustre: server umount lustre-MDT0001 complete [ 396.642148] Lustre: server umount lustre-OST0000 complete [ 402.143536] Lustre: server umount lustre-OST0001 complete [ 410.837920] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 424.003050] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 468.109794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 468.369356] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 468.412908] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 468.554928] Lustre: lustre-MDT0000: new disk, initializing [ 468.674826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 468.707419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 475.313417] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 488.783731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 488.924702] Lustre: 15098:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 488.983466] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 488.990146] Lustre: Skipped 1 previous similar message [ 489.104237] Lustre: lustre-MDT0001: new disk, initializing [ 489.230237] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 489.303106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 489.322944] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 494.814203] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 505.152555] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 511.871617] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 520.244491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 520.566657] Lustre: lustre-OST0000: new disk, initializing [ 520.571800] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 520.580714] Lustre: 17065:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 520.710804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 521.683919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 521.690429] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 521.894321] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 527.806203] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 539.306739] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 544.623708] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 544.848798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 550.115738] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 550.386612] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 557.550033] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 557.565310] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 557.592604] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 557.595645] Lustre: Skipped 1 previous similar message [ 558.759418] Lustre: server umount lustre-OST0000 complete [ 566.241501] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 566.259213] LustreError: Skipped 1 previous similar message [ 566.265782] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 566.279587] Lustre: Skipped 1 previous similar message [ 566.285915] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 569.504369] Lustre: server umount lustre-MDT0000 complete [ 572.902783] LustreError: 17372:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 572.929745] LustreError: 17372:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 573.881666] LustreError: 18406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783441582 with bad export cookie 4772611864199671811 [ 573.886806] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 573.903920] LustreError: 18406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 574.823869] Lustre: server umount lustre-MDT0001 complete [ 580.374491] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 12:26:27 (1783441587) [ 586.709599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 587.377242] LustreError: 19332:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 587.482444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 592.867139] LustreError: 19333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 593.002905] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 597.986171] LustreError: 19332:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 601.098794] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 606.076975] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 612.775210] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 617.855143] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 624.506556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 631.952963] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 638.867453] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 642.653187] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 652.769700] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 652.785265] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 652.804363] Lustre: Skipped 1 previous similar message [ 652.816178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 652.826105] Lustre: Skipped 2 previous similar messages [ 656.152194] Lustre: server umount lustre-MDT0000 complete [ 658.813499] LustreError: 21176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783441667 with bad export cookie 4772611864199672693 [ 658.816586] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 658.821513] LustreError: 21176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 658.912776] LustreError: 19327:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 658.921332] LustreError: 19327:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 659.115498] Lustre: server umount lustre-MDT0001 complete [ 666.787612] Lustre: server umount lustre-OST0000 complete [ 690.902931] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 702.538634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 702.795594] LustreError: 23716:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 702.896337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 702.898793] Lustre: Skipped 2 previous similar messages [ 706.983779] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 716.204558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 722.986936] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 727.892543] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 738.157472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 748.699750] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 759.704598] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 759.858766] Lustre: lustre-OST0001: new disk, initializing [ 759.863264] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 759.886611] Lustre: 26034:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 761.955506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 761.979444] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 762.190614] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 768.031344] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 777.820286] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 781.240802] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 793.066682] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 793.074266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 793.088894] Lustre: Skipped 5 previous similar messages [ 793.105971] Lustre: Skipped 7 previous similar messages [ 799.487711] Lustre: server umount lustre-MDT0000 complete [ 803.298315] LustreError: 23716:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 803.320991] LustreError: 23716:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 803.459683] LustreError: 23696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783441812 with bad export cookie 4772611864199673540 [ 803.469413] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 803.474331] LustreError: 23696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 803.874297] Lustre: server umount lustre-MDT0001 complete [ 817.844576] Lustre: server umount lustre-OST0000 complete [ 831.878483] Lustre: server umount lustre-OST0001 complete [ 850.032384] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 860.348933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 860.758243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 860.765223] Lustre: Skipped 3 previous similar messages [ 865.518265] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 870.888733] LustreError: 29053:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 870.913291] LustreError: 29053:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 874.185294] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 879.478257] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 882.540383] Lustre: 30201:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 889.909180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 897.566774] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 906.060352] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 912.341079] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 919.948776] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 922.915439] Lustre: 32080:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 930.787549] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 930.793268] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 930.813287] Lustre: Skipped 1 previous similar message [ 930.830875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 930.835552] Lustre: Skipped 4 previous similar messages [ 935.353028] Lustre: server umount lustre-MDT0000 complete [ 939.185789] LustreError: 29034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783441947 with bad export cookie 4772611864199674828 [ 939.201963] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 939.592520] Lustre: server umount lustre-MDT0001 complete [ 953.010891] Lustre: server umount lustre-OST0000 complete [ 967.411521] Lustre: server umount lustre-OST0001 complete [ 976.761238] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 12:33:03 (1783441983) [ 1008.909804] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1018.902407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1019.223799] LustreError: 34904:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1019.231756] LustreError: 34904:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 1024.078256] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1033.644994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1038.250393] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1040.522995] Lustre: 36051:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1047.120717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1054.067694] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1062.159090] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1070.233821] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1078.209518] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1081.926721] Lustre: 37933:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1090.531764] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1090.546188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1090.554029] Lustre: Skipped 5 previous similar messages [ 1090.573766] Lustre: Skipped 7 previous similar messages [ 1092.747886] Lustre: server umount lustre-MDT0000 complete [ 1096.216969] LustreError: 34886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783442105 with bad export cookie 4772611864199675941 [ 1096.223469] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1096.231805] LustreError: 34886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1096.468594] Lustre: server umount lustre-MDT0001 complete [ 1111.287187] Lustre: server umount lustre-OST0000 complete [ 1126.046806] Lustre: server umount lustre-OST0001 complete [ 1135.789292] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 12:35:42 (1783442142) [ 1152.286342] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1162.115247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1162.650140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1162.667627] Lustre: Skipped 7 previous similar messages [ 1167.672381] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1176.436597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1180.948040] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1183.473420] Lustre: 41223:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1189.458394] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1195.756342] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1203.397184] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1208.996640] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1215.968795] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1219.748520] Lustre: 43100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1220.484033] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1220.489803] Lustre: Skipped 1 previous similar message [ 1223.137305] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1223.152075] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1223.155992] Lustre: Skipped 3 previous similar messages [ 1223.168686] Lustre: Skipped 3 previous similar messages [ 1229.265799] Lustre: server umount lustre-MDT0000 complete [ 1232.497132] LustreError: 43102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783442241 with bad export cookie 4772611864199677047 [ 1232.502709] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1232.513769] LustreError: 43102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1238.957841] Lustre: server umount lustre-MDT0001 complete [ 1247.772928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1257.951656] LustreError: 3658:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9eaf020a5f80 x1870073574075008/t0(0) o250->MGC192.168.202.142@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1257.990563] Lustre: MGS: Client 274ace28-241b-47f1-806a-935609ece302 (at 0@lo) reconnecting [ 1258.014025] Lustre: MGC192.168.202.142@tcp: Connection restored to 0@lo (at 0@lo) [ 1262.090288] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1267.636250] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1270.367303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1275.122476] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1275.883775] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1275.891759] Lustre: Skipped 1 previous similar message [ 1283.497601] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1285.391836] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1290.170427] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1291.413850] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1296.355329] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1301.671488] Lustre: server umount lustre-MDT0000 complete [ 1304.545250] LustreError: 41587:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1304.564081] LustreError: 41587:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 26 previous similar messages [ 1304.991963] LustreError: 40054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783442313 with bad export cookie 4772611864199678174 [ 1305.001084] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1305.002131] LustreError: 40054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1305.021237] LustreError: Skipped 1 previous similar message [ 1305.307874] Lustre: server umount lustre-MDT0001 complete [ 1319.165827] Lustre: server umount lustre-OST0000 complete [ 1332.484355] Lustre: server umount lustre-OST0001 complete [ 1341.839428] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 12:39:08 (1783442348) [ 1358.555038] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1369.562046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1374.098621] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1382.579138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1387.331719] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1390.387034] Lustre: 48863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1396.977976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1404.690842] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1413.951503] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1420.876422] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1428.270164] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1432.162706] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1444.076270] Lustre: server umount lustre-MDT0000 complete [ 1444.323622] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1444.332590] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1444.344976] Lustre: Skipped 7 previous similar messages [ 1447.847892] LustreError: 47691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783442456 with bad export cookie 4772611864199678972 [ 1447.852544] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1447.859123] LustreError: 47691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1448.377709] Lustre: server umount lustre-MDT0001 complete [ 1461.366009] Lustre: server umount lustre-OST0000 complete [ 1475.122713] Lustre: server umount lustre-OST0001 complete [ 1482.768025] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 12:41:30 (1783442490) [ 1484.907172] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1486.515836] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 12:41:33 (1783442493) [ 1495.419427] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 1503.495044] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1527.679948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1528.219348] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1528.276945] Lustre: lustre-MDT0000: new disk, initializing [ 1528.440466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1532.318369] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1540.730554] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1546.834370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1547.081420] Lustre: lustre-OST0000: new disk, initializing [ 1547.084326] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1547.087594] Lustre: Skipped 1 previous similar message [ 1547.091324] Lustre: 54934:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1548.485776] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1548.492417] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1548.533926] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1552.266319] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1559.808744] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1561.485392] Lustre: DEBUG MARKER: create test files [ 1589.738274] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1589.743071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1589.770032] Lustre: Skipped 15 previous similar messages [ 1594.238294] Lustre: server umount lustre-OST0000 complete [ 1597.584314] Lustre: server umount lustre-MDT0000 complete [ 1606.956883] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 1609.713677] Key type lgssc unregistered [ 1610.010713] LNet: 56564:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1610.021549] LNetError: 56564:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1610.035588] LNet: Removed LNI 192.168.202.142@tcp [ 1610.908316] Key type .llcrypt unregistered [ 1610.912045] Key type ._llcrypt unregistered [ 1643.230806] Key type ._llcrypt registered [ 1643.236045] Key type .llcrypt registered [ 1643.446924] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1645.261844] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1645.341596] alg: No test for adler32 (adler32-zlib) [ 1646.735159] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 1647.172619] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 1648.975187] Key type lgssc registered [ 1650.296426] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1659.473515] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1659.492613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1661.096607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1665.916396] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1673.036174] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1679.612173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1679.851603] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1685.998089] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1686.132766] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1686.201764] Lustre: 59306:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1686.213555] Lustre: 59306:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1686.224627] Lustre: 59306:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1686.234802] Lustre: 59306:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1686.245218] Lustre: 59306:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1686.251571] Lustre: 59306:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1693.224820] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1699.586873] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1704.197508] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1711.589193] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1711.593763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1711.642151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1713.216066] Lustre: server umount lustre-OST0000 complete [ 1717.754739] Lustre: server umount lustre-MDT0000 complete [ 1729.661659] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 1732.567702] Key type lgssc unregistered [ 1733.045735] LNet: 60911:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1733.051357] LNetError: 60911:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1733.068841] LNet: Removed LNI 192.168.202.142@tcp [ 1733.874784] Key type .llcrypt unregistered [ 1733.877306] Key type ._llcrypt unregistered [ 1765.545325] Key type ._llcrypt registered [ 1765.547602] Key type .llcrypt registered [ 1765.650912] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1766.754779] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1766.828062] alg: No test for adler32 (adler32-zlib) [ 1767.863819] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 1767.994289] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 1769.687526] Key type lgssc registered [ 1771.217320] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1778.569873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1778.600691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1779.961744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1783.716936] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1792.018827] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1798.109869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1798.569925] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1804.726377] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1804.859115] Lustre: 63748:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1804.864491] Lustre: 63748:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1804.877713] Lustre: 63748:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1804.887152] Lustre: 63748:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1804.895760] Lustre: 63748:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1804.901862] Lustre: 63748:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1806.574180] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1814.812933] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1820.031328] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1830.370512] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1830.370640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1830.402576] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1834.101546] Lustre: server umount lustre-OST0000 complete [ 1838.644437] Lustre: server umount lustre-MDT0000 complete [ 1849.374319] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 1852.014477] Key type lgssc unregistered [ 1852.339644] LNet: 65304:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1852.349694] LNetError: 65304:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1852.375233] LNet: Removed LNI 192.168.202.142@tcp [ 1853.529522] Key type .llcrypt unregistered [ 1853.534499] Key type ._llcrypt unregistered [ 1876.041891] Key type ._llcrypt registered [ 1876.044402] Key type .llcrypt registered [ 1876.166834] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 1890.968902] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 1893.287789] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1893.321665] alg: No test for adler32 (adler32-zlib) [ 1894.657807] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 1894.942489] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 1896.583189] Key type lgssc registered [ 1897.872297] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1933.304566] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1933.320955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1934.725943] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1934.757428] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1934.830753] Lustre: lustre-MDT0000: new disk, initializing [ 1934.897625] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1934.918911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1940.855250] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1953.161843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1953.340783] Lustre: 68455:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1953.376909] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1953.379164] Lustre: Skipped 1 previous similar message [ 1953.421514] Lustre: lustre-MDT0001: new disk, initializing [ 1953.503144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1953.576452] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1953.582866] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1957.203537] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1967.223415] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1972.004739] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1979.737473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1979.981759] Lustre: lustre-OST0000: new disk, initializing [ 1979.984901] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1979.989664] Lustre: 70429:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1980.073733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1988.336075] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 1990.349635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1990.365545] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1990.544518] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2002.711361] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2009.762371] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2009.978666] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2015.442803] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2015.690698] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2019.297518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2019.304584] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2019.307122] Lustre: Skipped 1 previous similar message [ 2019.315829] Lustre: Skipped 1 previous similar message [ 2023.571438] Lustre: server umount lustre-OST0000 complete [ 2029.548936] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2029.551513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2029.557385] Lustre: Skipped 1 previous similar message [ 2029.568672] Lustre: Skipped 1 previous similar message [ 2033.861938] Lustre: server umount lustre-MDT0000 complete [ 2037.174540] LustreError: 68447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783443045 with bad export cookie 3212207698954553863 [ 2037.183464] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2037.821769] Lustre: server umount lustre-MDT0001 complete [ 2047.257515] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 12:50:54 (1783443054) [ 2058.913933] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 2095.322344] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 2105.143746] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 2140.527448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2140.793440] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2140.826542] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2140.898043] Lustre: lustre-MDT0000: new disk, initializing [ 2140.997695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2141.019970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2146.235265] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2156.616327] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2156.786621] Lustre: 76447:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2156.901950] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2156.907849] Lustre: Skipped 1 previous similar message [ 2157.014901] Lustre: lustre-MDT0001: new disk, initializing [ 2157.079831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2157.109056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2157.119341] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2161.200374] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2171.337862] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2176.334757] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2182.608421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2182.831911] Lustre: lustre-OST0000: new disk, initializing [ 2182.835297] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2182.840570] Lustre: 78416:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2182.938976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2188.339185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2188.354101] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2188.466273] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2189.236694] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2197.564970] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2202.107863] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2202.277994] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2207.011966] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2207.194566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2209.251980] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2209.255567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2209.269971] Lustre: Skipped 1 previous similar message [ 2209.284966] Lustre: Skipped 1 previous similar message [ 2214.374383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2214.380807] Lustre: Skipped 1 previous similar message [ 2215.089819] Lustre: server umount lustre-OST0000 complete [ 2218.471937] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2218.482531] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2218.494431] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2224.610965] LustreError: 76459:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2224.632945] LustreError: 76459:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2224.727806] Lustre: server umount lustre-MDT0000 complete [ 2228.589367] LustreError: 76440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783443237 with bad export cookie 3212207698954554745 [ 2228.597230] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2228.602952] LustreError: 76440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2228.946526] Lustre: server umount lustre-MDT0001 complete [ 2236.255158] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 12:54:03 (1783443243) [ 2242.407180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2242.807970] LustreError: 80827:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2242.822764] LustreError: 80827:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2242.876569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2247.170500] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2248.174411] LustreError: 80828:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2252.257317] LustreError: 80827:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2254.967498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2259.305288] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2266.296060] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2271.152396] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2276.865459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2277.101977] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2277.110895] Lustre: Skipped 1 previous similar message [ 2282.782557] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2289.340179] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2292.406367] Lustre: 83497:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 4ff0bbef-e4ee-4b9f-8717-f60f6576c72c at adminstrative request [ 2292.726253] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2292.735971] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2302.954856] Lustre: MGS: Client 4ff0bbef-e4ee-4b9f-8717-f60f6576c72c (at 0@lo) reconnecting [ 2302.962046] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x2c940de218f610eb to 0x2c940de218f6138b [ 2302.969649] Lustre: MGC192.168.202.142@tcp: Connection restored to 0@lo (at 0@lo) [ 2329.007861] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2329.199947] Lustre: lustre-OST0001: new disk, initializing [ 2329.202777] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2329.210278] Lustre: 83889:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2335.006962] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2336.394730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2336.407421] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2336.510374] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2343.461966] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2346.998890] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2347.005839] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2347.015046] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2347.015056] Lustre: Skipped 2 previous similar messages [ 2347.050615] Lustre: Skipped 2 previous similar messages [ 2352.260969] Lustre: server umount lustre-OST0001 complete [ 2357.218242] LustreError: 82646:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2357.222205] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2357.240863] LustreError: 82646:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2357.248866] Lustre: Skipped 1 previous similar message [ 2367.456337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2367.465796] LustreError: 82648:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2367.468858] Lustre: Skipped 8 previous similar messages [ 2367.491509] LustreError: 82648:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2370.015365] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2370.164394] Lustre: server umount lustre-OST0000 complete [ 2377.700762] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2377.716914] Lustre: Skipped 1 previous similar message [ 2379.947210] Lustre: server umount lustre-MDT0000 complete [ 2383.632786] LustreError: 80808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783443392 with bad export cookie 3212207698954556299 [ 2383.636495] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2383.642341] LustreError: 80808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2383.947552] Lustre: server umount lustre-MDT0001 complete [ 2392.025923] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 12:56:39 (1783443399) [ 2393.901855] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2395.853886] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 12:56:42 (1783443402) [ 2397.746172] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2399.508018] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 12:56:46 (1783443406) [ 2400.998171] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2402.890401] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 12:56:49 (1783443409) [ 2404.750846] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2406.487615] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 12:56:53 (1783443413) [ 2408.066649] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2413.605575] loop: module loaded [ 2413.619088] loop0: detected capacity change from 0 to 4294967296 [ 2413.663734] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2415.100390] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2418.924819] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2423.595655] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2449.221130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2449.284203] LustreError: 88014:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2449.292181] LustreError: 88014:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2449.306627] LustreError: 88014:0:(tgt_mount.c:2580:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2449.319942] LustreError: 88014:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2467.011715] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 2475.169499] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 2506.462223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2506.658971] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2506.693534] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2506.761243] Lustre: lustre-MDT0000: new disk, initializing [ 2506.825380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2506.828442] Lustre: Skipped 1 previous similar message [ 2506.837898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2510.484785] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2520.503568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2520.603572] Lustre: 90879:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2520.819087] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2524.768350] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2533.213897] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2538.189938] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2544.220255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2544.453192] Lustre: 92845:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2550.346488] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2553.423350] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2553.550685] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2558.970635] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2563.536652] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2563.730305] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2567.725735] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2567.918634] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2574.313518] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2574.318502] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2574.331658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2574.348688] Lustre: Skipped 1 previous similar message [ 2574.356683] Lustre: Skipped 3 previous similar messages [ 2575.689780] Lustre: server umount lustre-OST0000 complete [ 2585.294263] Lustre: server umount lustre-MDT0000 complete [ 2588.844284] LustreError: 90871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783443597 with bad export cookie 3212207698954556915 [ 2588.849923] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2588.855364] LustreError: 90871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2589.040783] Lustre: server umount lustre-MDT0001 complete [ 2596.855348] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 13:00:04 (1783443604) [ 2602.714865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2603.127778] LustreError: 95257:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2603.140393] LustreError: 95257:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2607.679254] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2614.848687] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2619.124903] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2625.200761] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2630.000783] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2635.471839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2635.726689] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2635.729984] Lustre: Skipped 4 previous similar messages [ 2640.794847] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2646.260226] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2650.887830] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2651.066809] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2655.259837] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2655.406264] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2661.120316] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2661.295392] Lustre: lustre-OST0001: new disk, initializing [ 2661.297985] Lustre: Skipped 2 previous similar messages [ 2661.301412] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2661.304175] Lustre: Skipped 3 previous similar messages [ 2661.312491] Lustre: 98639:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2667.339310] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2671.157034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2671.168478] Lustre: Skipped 2 previous similar messages [ 2671.173414] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2671.318559] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2674.640434] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2678.287531] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2678.470345] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2681.981221] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2682.121541] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2699.305639] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2718.219425] Lustre: Failing over lustre-MDT0000 [ 2718.605606] Lustre: server umount lustre-MDT0000 complete [ 2720.740892] LustreError: 97378:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.42@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2720.751128] LustreError: 97378:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2722.790190] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2722.792023] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2722.800270] LustreError: Skipped 1 previous similar message [ 2722.817850] Lustre: Skipped 5 previous similar messages [ 2739.168123] Lustre: 66678:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783443731/real 1783443731] req@000000009daa3716 x1870075425717632/t0(0) o400->MGC192.168.202.142@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1783443747 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2739.193313] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2740.894544] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2740.896313] LDISKFS-fs (dm-0): recovery complete [ 2740.909655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2748.388499] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x2c940de218f6de06 [ 2748.396262] Lustre: MGC192.168.202.142@tcp: Connection restored to 0@lo (at 0@lo) [ 2748.637687] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2751.649561] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2752.467902] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2752.473819] Lustre: 101459:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2754.034105] Lustre: 95258:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2754.048433] Lustre: 95258:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2754.057016] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2754.120225] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2754.439149] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2754.763593] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2754.771473] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2755.095088] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2755.099238] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2755.104414] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2755.108882] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2756.055334] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2756.062178] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 2756.392194] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2756.402352] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 2757.361564] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2757.371760] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 2758.327980] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2758.344148] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2758.663593] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2758.675687] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2761.527989] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2761.539235] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2762.498238] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2762.510819] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2762.831128] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2762.839106] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2769.809579] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2769.827839] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 2770.789449] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2770.794395] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 2771.121952] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2771.125236] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 2786.009887] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2786.023276] Lustre: 101486:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2786.977953] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2786.980943] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2787.295152] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2787.298349] LustreError: 101486:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2812.215139] Lustre: 101486:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2812.221793] Lustre: 101486:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2812.227940] Lustre: 101486:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 74a5975b-2452-4330-b113-68ed0d5db679@192.168.202.42@tcp [ 2812.246167] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2812.256916] Lustre: 101486:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@00000000eba14cd8 x1870075409798400/t0(8589934780) o35->74a5975b-2452-4330-b113-68ed0d5db679@192.168.202.42@tcp:196/0 lens 392/0 e 0 to 0 dl 1783443831 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2812.316912] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2812.335234] Lustre: lustre-MDT0000: Denying connection for new client 74a5975b-2452-4330-b113-68ed0d5db679 (at 192.168.202.42@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2812.353512] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2812.390173] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2812.393101] Lustre: Skipped 2 previous similar messages [ 2812.397174] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2812.447381] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2812.457333] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2825.711473] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2825.720067] Lustre: Skipped 5 previous similar messages [ 2831.731042] Lustre: server umount lustre-OST0000 complete [ 2835.424229] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2835.938395] LustreError: 97076:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2835.984996] LustreError: 97076:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2849.247541] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2849.396205] Lustre: server umount lustre-OST0001 complete [ 2853.349070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2853.353163] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2853.369742] Lustre: Skipped 4 previous similar messages [ 2858.713752] Lustre: server umount lustre-MDT0000 complete [ 2861.527158] LustreError: 95238:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783443870 with bad export cookie 3212207698954608134 [ 2861.528169] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2861.540974] LustreError: 95238:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2861.928551] Lustre: server umount lustre-MDT0001 complete [ 2868.708285] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 13:04:36 (1783443876) [ 2874.411947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2874.414787] Lustre: *** cfs_fail_loc=197, val=0*** [ 2889.951666] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2895.071538] LustreError: 103463:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.142@tcp: failed processing log, type 4: rc = -110 [ 2900.218243] Lustre: *** cfs_fail_loc=197, val=0*** [ 2900.224357] Lustre: Skipped 65 previous similar messages [ 2920.799875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2920.802915] Lustre: Skipped 2 previous similar messages [ 2926.174707] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2932.127470] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2933.722681] Lustre: server umount lustre-OST0000 complete [ 2940.064589] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 13:05:47 (1783443947) [ 2958.521972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2962.131634] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 2969.065156] LustreError: 105082:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2969.075315] LustreError: 105082:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2969.141649] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2972.530217] LustreError: 105112:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2972.537888] LustreError: 105112:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2972.544612] LustreError: 105112:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 14, retries 0, failed: rc = -5 [ 2972.781803] Lustre: server umount lustre-MDT0000 complete [ 3000.695963] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 3023.188934] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 3066.291366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3066.591055] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3066.651159] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3066.815805] Lustre: lustre-MDT0000: new disk, initializing [ 3066.942765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3072.181626] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3082.817293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3082.951891] Lustre: 109313:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3083.237804] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3089.559838] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3100.854289] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3107.021917] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3112.631123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3112.888287] Lustre: 111284:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3118.237347] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3119.617730] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3129.504831] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3134.800892] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3135.008182] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3140.211739] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3140.398891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3143.648342] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3143.665382] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3143.689267] Lustre: Skipped 1 previous similar message [ 3143.693278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3143.697470] Lustre: Skipped 11 previous similar messages [ 3148.670885] Lustre: server umount lustre-OST0000 complete [ 3158.904735] Lustre: server umount lustre-MDT0000 complete [ 3162.966648] LustreError: 110109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783444171 with bad export cookie 3212207698954618830 [ 3162.967259] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3162.984962] LustreError: 110109:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3163.396546] Lustre: server umount lustre-MDT0001 complete [ 3171.353721] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 13:09:38 (1783444178) [ 3180.350126] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 3183.972368] Key type lgssc unregistered [ 3184.473218] LNet: 113771:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3184.484252] LNetError: 113771:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3184.503405] LNet: Removed LNI 192.168.202.142@tcp [ 3185.705451] Key type .llcrypt unregistered [ 3185.708838] Key type ._llcrypt unregistered [ 3209.593986] Key type ._llcrypt registered [ 3209.601532] Key type .llcrypt registered [ 3209.898789] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 3230.355495] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 3232.073720] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3232.136501] alg: No test for adler32 (adler32-zlib) [ 3233.402072] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 3233.692811] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 3235.511312] Key type lgssc registered [ 3236.635962] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3280.663129] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3280.729171] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3282.074212] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3282.128679] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3282.233510] Lustre: lustre-MDT0000: new disk, initializing [ 3282.343474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3282.377541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3288.305315] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3301.041838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3301.457184] Lustre: lustre-OST0000: new disk, initializing [ 3301.463756] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3301.467756] Lustre: Skipped 1 previous similar message [ 3301.475347] Lustre: 117946:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3301.549922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3308.368784] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3309.174155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3309.180742] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3309.221259] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3319.116593] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3319.279521] Lustre: lustre-OST0001: new disk, initializing [ 3319.287086] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3319.296594] Lustre: 118812:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3319.355117] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3326.201837] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3329.667116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3329.672976] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3329.709145] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3339.746366] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3339.783292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3339.800675] Lustre: Skipped 1 previous similar message [ 3342.483748] Lustre: server umount lustre-MDT0000 complete [ 3350.573383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3356.509855] LustreError: 117132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783444365 with bad export cookie 6690519372101069778 [ 3356.512769] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3356.535939] LustreError: 117132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3356.638457] Lustre: server umount lustre-OST0000 complete [ 3360.756568] Lustre: server umount lustre-OST0001 complete [ 3369.514674] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 13:12:56 (1783444376) [ 3393.731308] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 13:13:19 (1783444399) [ 3409.746685] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 3438.542151] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 13:14:05 (1783444445) [ 3486.069560] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 3496.192320] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 3538.549532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3538.777221] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3538.802452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3538.866351] Lustre: lustre-MDT0000: new disk, initializing [ 3538.936135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3538.950734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3544.016741] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3555.078211] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3555.298784] Lustre: 126595:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3555.357831] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3555.364953] Lustre: Skipped 1 previous similar message [ 3555.454599] Lustre: lustre-MDT0001: new disk, initializing [ 3555.594646] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3555.627707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3555.649230] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3561.340338] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3570.348840] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3575.808393] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3583.242612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3583.529531] Lustre: lustre-OST0000: new disk, initializing [ 3583.533572] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3583.546538] Lustre: 128563:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3583.641532] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3590.181300] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3592.820039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3592.827772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3592.907387] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3600.248813] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3605.028893] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3605.271417] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3610.110472] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3610.415973] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3613.667790] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3613.670934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3613.685391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3613.700779] Lustre: Skipped 2 previous similar messages [ 3613.712853] Lustre: Skipped 1 previous similar message [ 3618.801029] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3618.811920] Lustre: Skipped 1 previous similar message [ 3619.017625] Lustre: server umount lustre-OST0000 complete [ 3623.905864] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3623.910680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3623.919957] Lustre: Skipped 1 previous similar message [ 3623.929517] Lustre: Skipped 1 previous similar message [ 3629.024178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3629.036652] Lustre: Skipped 1 previous similar message [ 3629.923866] Lustre: server umount lustre-MDT0000 complete [ 3634.215406] LustreError: 126587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783444643 with bad export cookie 6690519372101071003 [ 3634.216683] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3634.228298] LustreError: 126587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3634.644246] Lustre: server umount lustre-MDT0001 complete [ 3644.108453] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 13:17:31 (1783444651) [ 3651.217989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3651.825898] LustreError: 130974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3651.838889] LustreError: 130974:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3652.018661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3657.100530] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3657.194537] LustreError: 130975:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3661.316193] LustreError: 130974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3666.402312] LustreError: 130975:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3666.992513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3673.623196] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3684.885798] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3691.511398] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3699.721565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3700.164927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3700.174890] Lustre: Skipped 1 previous similar message [ 3707.930405] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3716.319177] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3726.835466] Lustre: *** cfs_fail_loc=159, val=0*** [ 3743.271136] Lustre: lustre-MDT0000: Client 9d2b8eb0-0ace-4f95-a0e1-b7073b516f2e (at 192.168.202.42@tcp) reconnecting [ 3745.105954] Lustre: *** cfs_fail_loc=159, val=0*** [ 3745.107777] Lustre: Skipped 5 previous similar messages [ 3760.693790] Lustre: lustre-MDT0000: Client 9d2b8eb0-0ace-4f95-a0e1-b7073b516f2e (at 192.168.202.42@tcp) reconnecting [ 3767.777197] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3767.802966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3767.808781] Lustre: Skipped 1 previous similar message [ 3781.087749] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3781.302054] Lustre: server umount lustre-OST0000 complete [ 3788.258371] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3788.289844] Lustre: Skipped 1 previous similar message [ 3788.313864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3788.324765] Lustre: Skipped 5 previous similar messages [ 3791.909789] Lustre: server umount lustre-MDT0000 complete [ 3793.385279] LustreError: 133701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3793.401772] LustreError: 133701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3795.483513] LustreError: 133703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783444804 with bad export cookie 6690519372101071885 [ 3795.490669] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3795.492991] LustreError: 133703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3796.014843] Lustre: server umount lustre-MDT0001 complete [ 3809.608789] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 3814.967272] Key type lgssc unregistered [ 3815.696220] LNet: 134759:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3815.705017] LNetError: 134759:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3815.721443] LNet: Removed LNI 192.168.202.142@tcp [ 3817.459318] Key type .llcrypt unregistered [ 3817.465114] Key type ._llcrypt unregistered [ 3847.198567] Key type ._llcrypt registered [ 3847.201071] Key type .llcrypt registered [ 3847.283624] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 3848.315823] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3848.382203] alg: No test for adler32 (adler32-zlib) [ 3849.670719] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 3849.933150] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 3851.568826] Key type lgssc registered [ 3853.414741] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3865.199397] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3865.220934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3866.732174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3873.623028] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3876.323711] LustreError: 136178:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3876.351810] LustreError: 136178:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3881.453936] LustreError: 136177:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3886.005050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3886.339210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3893.286164] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3904.448517] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3910.637493] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3917.176554] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3917.427885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3924.490215] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 3932.372697] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3936.941204] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3947.988215] Lustre: *** cfs_fail_loc=159, val=0*** [ 3963.442594] Lustre: lustre-MDT0000: Client b6e46242-6340-4bec-abf4-b0d0c08d68cd (at 192.168.202.42@tcp) reconnecting [ 3965.653206] Lustre: *** cfs_fail_loc=159, val=0*** [ 3981.849110] Lustre: lustre-MDT0001: Client b6e46242-6340-4bec-abf4-b0d0c08d68cd (at 192.168.202.42@tcp) reconnecting [ 3983.458700] Lustre: *** cfs_fail_loc=159, val=0*** [ 3983.461569] Lustre: Skipped 3 previous similar messages [ 3999.753697] Lustre: lustre-MDT0001: Client b6e46242-6340-4bec-abf4-b0d0c08d68cd (at 192.168.202.42@tcp) reconnecting [ 4015.584600] Lustre: 139254:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4018.734885] Lustre: *** cfs_fail_loc=159, val=0*** [ 4018.738040] Lustre: Skipped 3 previous similar messages [ 4034.610593] Lustre: lustre-MDT0000: Client 8bd28d05-7e0a-4e64-886a-df4c4d2896ce (at 192.168.202.42@tcp) reconnecting [ 4036.124791] Lustre: *** cfs_fail_loc=159, val=0*** [ 4036.128098] Lustre: Skipped 14 previous similar messages [ 4051.499040] Lustre: lustre-MDT0000: Client 8bd28d05-7e0a-4e64-886a-df4c4d2896ce (at 192.168.202.42@tcp) reconnecting [ 4052.614645] Lustre: 139549:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4057.066776] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4057.072198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4057.074390] LustreError: Skipped 2 previous similar messages [ 4057.107371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4058.517545] Lustre: server umount lustre-OST0000 complete [ 4065.251486] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4065.255529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4065.276803] Lustre: Skipped 1 previous similar message [ 4069.369131] Lustre: server umount lustre-MDT0000 complete [ 4070.370596] LustreError: 139376:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4070.408609] LustreError: 139376:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4073.793232] LustreError: 136159:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783445082 with bad export cookie 15407987933156837431 [ 4073.805311] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4073.808922] LustreError: 136159:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4084.504328] Lustre: server umount lustre-MDT0001 complete [ 4096.524183] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 4099.634703] Key type lgssc unregistered [ 4099.977435] LNet: 140603:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4099.998952] LNetError: 140603:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4100.017076] LNet: Removed LNI 192.168.202.142@tcp [ 4101.287546] Key type .llcrypt unregistered [ 4101.290118] Key type ._llcrypt unregistered [ 4129.391406] Key type ._llcrypt registered [ 4129.393773] Key type .llcrypt registered [ 4129.527529] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 4131.048286] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4131.069530] alg: No test for adler32 (adler32-zlib) [ 4132.229759] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 4132.508126] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 4134.215546] Key type lgssc registered [ 4135.702985] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4144.579385] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4144.602595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4146.246062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4151.696884] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4155.361853] LustreError: 142020:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4155.387124] LustreError: 142020:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4160.487788] LustreError: 142019:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4160.670633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4161.266862] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4166.363607] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4173.644452] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4180.634555] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4187.544643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4187.939901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4191.039891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4197.187222] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4205.399341] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4206.278208] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4212.180422] Lustre: 144600:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4214.602658] Lustre: 144649:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4215.878164] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4216.703989] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4219.999851] Lustre: 144795:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4220.856830] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4221.936588] Lustre: 144894:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4226.017312] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4226.021905] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4226.022016] Lustre: Skipped 1 previous similar message [ 4226.036021] Lustre: Skipped 1 previous similar message [ 4231.162047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4231.174473] Lustre: Skipped 1 previous similar message [ 4232.078782] Lustre: server umount lustre-OST0000 complete [ 4237.792853] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4237.809960] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4237.819778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4241.376936] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4241.377871] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4241.385234] Lustre: Skipped 1 previous similar message [ 4242.890919] Lustre: server umount lustre-MDT0000 complete [ 4246.499286] LustreError: 142019:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4246.533952] LustreError: 142019:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4249.149321] LustreError: 142001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783445257 with bad export cookie 17074299713131099749 [ 4249.157212] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4249.157925] LustreError: 142001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4249.460585] Lustre: server umount lustre-MDT0001 complete [ 4264.231518] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 4267.073850] Key type lgssc unregistered [ 4267.376658] LNet: 145956:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4267.383721] LNetError: 145956:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4268.454859] LNet: Removed LNI 192.168.202.142@tcp [ 4269.467361] Key type .llcrypt unregistered [ 4269.474060] Key type ._llcrypt unregistered [ 4294.546729] Key type ._llcrypt registered [ 4294.548756] Key type .llcrypt registered [ 4294.635824] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 4297.895812] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4298.100482] alg: No test for adler32 (adler32-zlib) [ 4299.429583] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 4300.135419] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 4302.007758] Key type lgssc registered [ 4303.881625] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4313.477791] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4313.501967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4315.052253] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4319.220580] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4324.332341] LustreError: 147372:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4324.344635] LustreError: 147372:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4326.628061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4327.257513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4331.911826] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4338.005587] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4341.235929] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4350.306048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4351.127416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4352.206851] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4359.376334] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4362.257920] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4373.468854] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4380.173678] Lustre: *** cfs_fail_loc=159, val=0*** [ 4380.181130] Lustre: Skipped 1 previous similar message [ 4395.578671] Lustre: lustre-MDT0000: Client e29ca2b4-354f-42dc-b04e-86ed6b780dde (at 192.168.202.42@tcp) reconnecting [ 4403.684395] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4403.692577] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4403.718192] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4405.730778] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4405.744502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4405.752818] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4405.809044] Lustre: Skipped 1 previous similar message [ 4409.517128] Lustre: server umount lustre-OST0000 complete [ 4415.968256] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4415.972473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4415.993271] Lustre: Skipped 1 previous similar message [ 4421.092629] LustreError: 147372:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4421.415893] Lustre: server umount lustre-MDT0000 complete [ 4426.084417] LustreError: 147353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783445434 with bad export cookie 15087625073797285171 [ 4426.120430] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4426.648263] Lustre: server umount lustre-MDT0001 complete [ 4438.704067] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 4441.916870] Key type lgssc unregistered [ 4442.302118] LNet: 151037:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4442.323843] LNetError: 151037:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4442.352342] LNet: Removed LNI 192.168.202.142@tcp [ 4443.457232] Key type .llcrypt unregistered [ 4443.459583] Key type ._llcrypt unregistered [ 4468.209447] Key type ._llcrypt registered [ 4468.212209] Key type .llcrypt registered [ 4468.331662] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 4469.872580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4470.047351] alg: No test for adler32 (adler32-zlib) [ 4471.116243] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 4471.403631] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 4473.169690] Key type lgssc registered [ 4474.659648] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4482.804473] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4482.823659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4484.316402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4491.069640] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4494.823551] LustreError: 152454:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4494.859293] LustreError: 152454:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4499.937586] LustreError: 152453:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4505.061725] LustreError: 152454:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4505.639563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4506.051893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4511.507461] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4519.360846] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4525.818983] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4533.129401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4533.563409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4541.585198] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4549.999835] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4550.970870] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4555.077114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4574.360684] Lustre: lustre-OST0000: evicting d08f5665-268b-493a-9c65-b5cf1c347e5f (at 192.168.202.42@tcp) by administrative request [ 4574.369888] Lustre: Skipped 1 previous similar message [ 4606.447129] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4606.456270] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4606.462967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4606.488426] Lustre: Skipped 1 previous similar message [ 4609.244529] Lustre: server umount lustre-OST0000 complete [ 4613.601318] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4613.612488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4613.649493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4613.660029] Lustre: Skipped 1 previous similar message [ 4616.674921] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4616.676874] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4616.714708] Lustre: Skipped 1 previous similar message [ 4619.575879] Lustre: server umount lustre-MDT0000 complete [ 4621.795940] LustreError: 152453:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4621.819466] LustreError: 152453:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4624.786291] LustreError: 152433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783445633 with bad export cookie 12715117626331852321 [ 4624.822496] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4625.244343] Lustre: server umount lustre-MDT0001 complete [ 4636.410848] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 4640.237561] Key type lgssc unregistered [ 4640.521800] LNet: 156537:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4640.526410] LNetError: 156537:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4640.548142] LNet: Removed LNI 192.168.202.142@tcp [ 4641.292135] Key type .llcrypt unregistered [ 4641.293991] Key type ._llcrypt unregistered [ 4731.591846] Key type ._llcrypt registered [ 4731.604589] Key type .llcrypt registered [ 4731.878228] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 4733.410633] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4733.455192] alg: No test for adler32 (adler32-zlib) [ 4734.499548] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 4734.722441] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 4736.503953] Key type lgssc registered [ 4737.356815] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4745.881437] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4745.896511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4747.352739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4750.614816] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4757.485927] LustreError: 159028:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4757.508708] LustreError: 159028:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4760.520691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4761.058556] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4766.398768] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4775.570871] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4781.123659] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4787.281372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4787.611912] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4792.751707] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4792.951964] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4800.763373] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4806.068230] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4814.825123] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4814.833970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4814.861506] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4816.353613] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4816.367251] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4821.007658] Lustre: server umount lustre-OST0000 complete [ 4826.601073] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4826.607978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4826.615895] Lustre: Skipped 1 previous similar message [ 4826.625401] Lustre: Skipped 1 previous similar message [ 4831.712697] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4831.723872] Lustre: Skipped 2 previous similar messages [ 4836.833049] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4836.837036] Lustre: Skipped 1 previous similar message [ 4839.391268] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4839.650638] Lustre: server umount lustre-MDT0000 complete [ 4841.956400] LustreError: 159028:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4841.967073] LustreError: 159028:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4842.699987] LustreError: 159008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783445851 with bad export cookie 5256403839147534820 [ 4842.703567] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4842.705825] LustreError: 159008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4842.982285] Lustre: server umount lustre-MDT0001 complete [ 4855.854791] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 4859.706677] Key type lgssc unregistered [ 4860.319878] LNet: 162590:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4860.336520] LNetError: 162590:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4860.373527] LNet: Removed LNI 192.168.202.142@tcp [ 4861.164608] Key type .llcrypt unregistered [ 4861.173537] Key type ._llcrypt unregistered [ 4886.072494] Key type ._llcrypt registered [ 4886.079826] Key type .llcrypt registered [ 4886.212740] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 4887.503873] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4887.545696] alg: No test for adler32 (adler32-zlib) [ 4888.614301] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 4888.824624] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 4890.487181] Key type lgssc registered [ 4891.783772] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4906.311538] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 13:38:33 (1783445913) [ 4919.825273] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 4927.630415] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 4961.095953] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4961.129737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4962.534368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4962.577601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4962.716425] Lustre: lustre-MDT0000: new disk, initializing [ 4962.887826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4962.914155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4967.035876] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4976.576733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4976.759399] Lustre: 166852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4976.812254] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4976.815737] Lustre: Skipped 1 previous similar message [ 4976.876367] Lustre: lustre-MDT0001: new disk, initializing [ 4976.961723] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4977.002662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4977.013711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4981.823705] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 4990.735609] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4995.607393] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5001.292072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5001.550472] Lustre: lustre-OST0000: new disk, initializing [ 5001.554731] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5001.561233] Lustre: 168819:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5001.639905] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5006.405618] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 5009.527914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5009.538103] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5014.078367] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5015.032688] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5025.253877] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5025.254093] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5025.268413] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5025.277466] Lustre: Skipped 1 previous similar message [ 5025.286812] Lustre: Skipped 1 previous similar message [ 5028.342666] Lustre: server umount lustre-OST0000 complete [ 5033.446683] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5033.460974] LustreError: Skipped 1 previous similar message [ 5033.466356] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5033.478393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5035.491628] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5035.501659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5035.514442] Lustre: Skipped 1 previous similar message [ 5040.612105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5040.617845] Lustre: Skipped 1 previous similar message [ 5045.735639] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5045.739970] Lustre: Skipped 1 previous similar message [ 5046.239130] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5046.438077] Lustre: server umount lustre-MDT0000 complete [ 5049.154686] LustreError: 168818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783446057 with bad export cookie 18305959005615261142 [ 5049.159165] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5049.167737] LustreError: 168818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5049.413540] Lustre: server umount lustre-MDT0001 complete [ 5059.099048] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 5061.046793] Key type lgssc unregistered [ 5061.320957] LNet: 170725:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5061.326170] LNetError: 170725:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5062.374860] LNet: Removed LNI 192.168.202.142@tcp [ 5063.023485] Key type .llcrypt unregistered [ 5063.025635] Key type ._llcrypt unregistered [ 5080.828357] Key type ._llcrypt registered [ 5080.836080] Key type .llcrypt registered [ 5080.920500] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 5081.917508] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5081.942322] alg: No test for adler32 (adler32-zlib) [ 5082.887118] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 5083.106170] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 5084.751186] Key type lgssc registered [ 5085.568341] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5091.413028] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5091.426946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5092.856439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5096.968393] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 5103.072317] LustreError: 172119:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5103.086330] LustreError: 172119:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5103.779874] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5104.104340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5107.875410] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 5113.847665] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5118.373932] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5124.240877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5124.592862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5129.609388] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 5135.413264] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5137.902049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5146.039328] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5146.059202] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5146.073369] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5146.080249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.097429] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5147.148051] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5147.173485] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5147.184542] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5147.189690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.209951] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5148.056830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.848312] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5148.857675] Lustre: Skipped 1 previous similar message [ 5148.870617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5148.875925] Lustre: Skipped 1 previous similar message [ 5148.877675] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5148.879951] Lustre: Skipped 1 previous similar message [ 5148.881564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.885899] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5148.890782] Lustre: Skipped 1 previous similar message [ 5149.485639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.239556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.880410] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5150.883316] Lustre: Skipped 2 previous similar messages [ 5150.902278] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5150.914173] Lustre: Skipped 2 previous similar messages [ 5150.917584] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5150.922607] Lustre: Skipped 2 previous similar messages [ 5150.926450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.939664] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5150.942788] Lustre: Skipped 2 previous similar messages [ 5151.687406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.411195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.134905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.916895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.842777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.607894] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5155.612880] Lustre: Skipped 5 previous similar messages [ 5155.633168] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5155.638236] Lustre: Skipped 5 previous similar messages [ 5155.640824] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5155.649861] Lustre: Skipped 5 previous similar messages [ 5155.654563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.662986] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5155.667918] Lustre: Skipped 5 previous similar messages [ 5156.545970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.304970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.206801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.868629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.797537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.573673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.475346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.229585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.932716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.752120] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5163.759597] Lustre: Skipped 9 previous similar messages [ 5163.774693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5163.797321] Lustre: Skipped 9 previous similar messages [ 5163.799749] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5163.803957] Lustre: Skipped 9 previous similar messages [ 5163.810200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.816558] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5163.820043] Lustre: Skipped 9 previous similar messages [ 5164.699476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.439389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.337423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.016994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.751648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.433571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.120527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.779958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.500700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.296883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.915898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.567374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.302088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.006905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.695261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.466331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.172614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.897607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.603606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.300232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.994610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.586841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.150416] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5180.154749] Lustre: Skipped 22 previous similar messages [ 5180.166871] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5180.174180] Lustre: Skipped 22 previous similar messages [ 5180.176854] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5180.179805] Lustre: Skipped 22 previous similar messages [ 5180.182505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.187898] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5180.196669] Lustre: Skipped 22 previous similar messages [ 5180.840715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.525254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.138787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.806393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.365213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.065382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.645478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.286738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.839418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.576456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.189689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.868965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.401582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.848238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.602773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.278267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.056458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.613334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.181256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.851081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.482839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.145926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.930875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.588647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.225933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.835591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.527696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.139953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.761953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.311523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.005636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.700789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.408206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.063755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.747027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.375538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.014696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.680690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.300405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.910884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.537295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.282791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.030326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.747456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.342201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.910452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.615484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.359879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.089869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.708078] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5212.714897] Lustre: Skipped 49 previous similar messages [ 5212.729794] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5212.742901] Lustre: Skipped 49 previous similar messages [ 5212.745741] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5212.750970] Lustre: Skipped 49 previous similar messages [ 5212.754868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.765617] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5212.771148] Lustre: Skipped 49 previous similar messages [ 5213.410187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.998229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.777163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.474064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.191214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.833847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.561324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.163241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.846252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.416037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.095722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.716790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.368965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.931980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.569789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.323499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.970144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.765564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.573648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.527368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.206995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.986640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.643989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.451805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.256480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.041463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.764754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.417066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.099172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.879906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.529528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.198812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.733285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.343279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.928732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.474759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.038696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.604495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.245658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.870794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.474520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.139412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.713441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.301427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.963322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.631841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.158618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.768449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.364418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.995099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.564621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.253349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.799870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.394870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.927529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.662531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.271644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.262127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.973576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.757107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.537595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.209556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.917723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.667247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.294244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.954881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.633683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.310840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.950708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.710966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.398910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.234426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.018759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.767812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.548969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.448392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.093418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.730336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.559937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.130614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.811497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.361039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.936792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.612639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.341686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.012080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.706763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.336263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.986457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.625894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.233935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.844185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.459041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.165290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.895858] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5276.897850] Lustre: Skipped 94 previous similar messages [ 5276.905493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.913086] Lustre: Skipped 94 previous similar messages [ 5276.917580] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.920668] Lustre: Skipped 94 previous similar messages [ 5276.923126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.931255] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.934588] Lustre: Skipped 94 previous similar messages [ 5277.694129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.457300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.249851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.024664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.740783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.519266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.112966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.949481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.561311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.267743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.868671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.596326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.482634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.237407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.248352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.014746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.951578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.729441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.513509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.366220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.967582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.805954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.456001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.235231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.924367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.615820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.192697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.886457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.617173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.435537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.229331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.891771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.664399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.410443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.221939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.035958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.067099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.787810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.591462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.368241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.036439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.709780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.437907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.962043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.556308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.149805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.727710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.325554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.930736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.684952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.219854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.886378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.530782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.156715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.702289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.316526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.023416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.778754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.440645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.192209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.818311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.653682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.412912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.149846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.893261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.793679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.503167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.226255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.053311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.699132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.381277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.967848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.635213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.330349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.072024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.942923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.840963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.590422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.476462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.277575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.987887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.656069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.302371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.934096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.660776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.235481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.944458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.797515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.520164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.130411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.894473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.550686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.191508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.829022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.491553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.300638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.906362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.719519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.432791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.209492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.012461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.804110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.535579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.141690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.831946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.506285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.123024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.724626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.348380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.002858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.597259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.276758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.936584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.561380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.205363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.900873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.465105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.121679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.763030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.385222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.021255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.741353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.354479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.922564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.566193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.176245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.717730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.372596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.986863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.562722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.141521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.701149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.292764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.900297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.418993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.034729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.725111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.322762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.924252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.551365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.106243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.767160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.286755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.948405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.537283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.192257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.745559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.350660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.935475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.607669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.219162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.843200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.490421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.079975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.644773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.236540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.782536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.527842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.169168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.875143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.631478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.348450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.081159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.991528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.682205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.374698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.991489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.846642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.667091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.347648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.987854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.603100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.224378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.820744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.514692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.203432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.846296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.470058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.069383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.678310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.414857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.124566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.805819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.490271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.178324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.773487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.409267] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5405.412565] Lustre: Skipped 186 previous similar messages [ 5405.425880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5405.432761] Lustre: Skipped 186 previous similar messages [ 5405.434916] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5405.438950] Lustre: Skipped 186 previous similar messages [ 5405.440675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.445550] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5405.449367] Lustre: Skipped 186 previous similar messages [ 5406.027951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.642960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.256383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.859199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.471562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.012976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.688860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.241586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.865931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.527458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.252460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.892643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.582080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.216764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.923571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.620588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.341759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.039636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.632666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.367254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.961616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.616738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.271735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.828226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.339219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.897169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.457366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.090850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.622782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.306847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.946562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.578566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.187660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.822897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.376329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.968799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.496214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.139503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.819688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.441377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.029324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.594090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.126628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.773222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.459410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.249484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.887345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.564994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.056559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.647842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.208301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.854242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.494862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.010109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.738458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.259489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.817285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.539051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.228605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.005838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.650674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.306896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.044830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.744874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.572979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.523348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.319130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.555227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.557935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.521980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.046835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.653305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.488041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.484216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.657245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.314212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.903167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.286866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.060082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.943392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.909975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.746632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.708733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.782332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.532283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.166258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.797824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.358918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.023759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.941903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.703937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.646685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.488261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.121488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.719655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.330385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.962301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.577985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.164593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.826305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.510144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.160274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.850041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.493386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.213871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.967605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.731563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.399755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.083689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.703352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.380236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.891377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.447901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.019719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.583293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.185805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.766493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.332447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.936521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.560925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.212455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.864675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.509291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.028445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.548365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.259035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.840159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.539831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.197030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.932851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.653327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.370095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.071950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.822341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.548310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.323662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.992507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.668086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.302756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.970833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.517748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.129149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.673867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.287887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.871835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.510621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.051838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.733530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.318237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.017922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.638984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.300611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.067835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.805352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.569435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.365460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.999976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.763364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.612685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.351181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.959795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.724224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.586131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.343204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.112381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.803571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.456461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.136249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.724126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.466474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.218395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.878513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.499907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.244697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.912866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.611814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.348438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.146085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.021464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.797915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.602413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.396706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.318031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.084878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.782421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.539601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.388363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.154722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.939609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.801402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.626503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.555393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.362462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.168856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.844934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.777690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.794713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.800074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.461545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.239859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.838345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.490285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.233333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.945279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.616495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.257818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.879545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.477249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.092818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.734493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.270530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.935669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.489637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.224131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.887515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.625070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.351991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.970669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.733889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.490479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.151356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.022573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.787854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.557659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.314595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.139749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.850338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.515405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.082485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.111665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.112700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.053606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.825353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.636569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.354565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.056545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.657357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.325442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.006341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.834595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.498072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.256916] LustreError: 172139:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5574.261908] LustreError: 172139:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5574.281550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.987803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.656273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.336915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.126074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.814092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.563192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.287415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.088577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.734912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.400355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.029889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.712397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.320913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.002137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.688968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.295345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.872369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.445341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.056470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.609318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.250179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.880925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.443910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.031367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.694461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.280368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.992733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.641137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.183480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.750285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.401857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.054794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.701781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.287645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.925487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.657645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.305651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.973827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.675402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.372811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.924384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.527692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.270588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.006599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.767522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.494465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.187720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.923636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.635935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.351142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.023765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.756675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.493847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.308315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.936638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.690215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.402981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.228835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.990358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.814863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.626160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.414071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.220117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.848069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.567307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.252461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.920660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.604266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.286739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.895766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.654956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.274085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.873466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.403695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.116628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.799398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.453903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.180810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.869532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.519784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.250853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.969986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.671562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.405348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.051087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.655917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.219155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.813520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.444194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.044532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.657168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.442464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.012580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.644855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.267635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.948216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.558402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.225393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.944380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.660346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.330346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.084055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.812309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.574339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.186547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.887676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.685385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.491723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.364697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.081172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.792321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.446665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.949194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.552346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.138209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.801765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.493184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.107297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.679764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.281319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.938220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.722609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.479535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.188610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.877291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.557848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.317615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.059727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.654505] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5661.668786] Lustre: Skipped 370 previous similar messages [ 5661.684408] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5661.698975] Lustre: Skipped 370 previous similar messages [ 5661.706698] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5661.711733] Lustre: Skipped 370 previous similar messages [ 5661.720843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.737210] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5661.746306] Lustre: Skipped 370 previous similar messages [ 5662.456658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.118393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.836502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.451188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.114382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.894021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.588284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.260652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.986815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.664300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.496868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.193144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.016274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.802662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.573888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.353921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.178146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.867839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.544773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.247320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.968189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.562632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.224231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.850244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.538858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.221644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.096877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.795208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.461669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.189542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.009353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.706465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.371340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.011766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.758296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.464459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.162321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.851251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.573829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.257625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.999724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.737884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.448984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.083948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.662164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.253810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.791473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.419163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.110229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.648604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.303382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.942890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.588533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.284059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.051782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.664880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.373847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.996659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.563112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.132549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.745842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.390727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.983070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.538659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.103707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.578700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.204366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.821628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.441089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.970560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.595365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.245797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.956792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.577424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.273233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.075709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.758927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.463197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.214990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.795740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.551240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.182708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.807106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.427478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.087800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.708207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.316386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.896101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.520519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.188359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.842284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.404471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.957383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.550807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.143414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.923025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.606101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.182421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.799102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.354804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.975644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.574556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.149528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.658333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.247244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.776209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.295620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.908563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.644856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.278283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.890221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.502126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.093692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.621530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.198489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.710640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.218772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.725565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.316500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.795876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.467516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.065111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.573278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.184081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.787424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.440744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.069183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.683887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.310770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.933155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.525085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.258954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.920885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.557921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.070337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.560778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.052352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.626404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.215301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.813869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.452913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.110385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.721305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.451086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.045288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.753138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.395850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.063653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.659647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.243103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.788661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.353223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.901385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.479475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.054077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.637191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.110425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.638841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.137687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.642681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.220047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.805868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.380629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.049356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.605336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.212241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.745558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.372897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.023457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.560127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.120585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.673440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.186029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.705561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.191541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.728000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.269612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.720282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.306584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.864723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.412067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.031966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.721388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.362210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.931484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.592351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.237632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.813384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.514585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.262872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.875176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.598576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.370060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.030891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.680221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.364724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.984243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.660379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.292875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.867943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.445625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.055160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.646933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.196156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.740990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.409387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.008590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.638579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.317165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.857122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.431391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.120566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.774150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.338513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.919423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.523400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.117438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.676887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.438218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.117226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.770490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.385444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.026074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.607458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.297915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.067898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.877194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.493843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.188031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.784967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.396982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.098571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.790263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.474311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.137488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.653100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.285883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.906619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.475304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.050959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.692879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.367539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.030642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.673943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.307982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.817447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.329863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.962230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.639605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.287585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.914205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.698411] LustreError: 173943:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6100.705946] LustreError: 172139:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6351.328133] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6351.340941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6351.354309] Lustre: Skipped 251 previous similar messages [ 6351.366490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6352.352521] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6352.365827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6355.424644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6355.434646] Lustre: Skipped 1 previous similar message [ 6357.371757] Lustre: server umount lustre-OST0000 complete [ 6360.551617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6360.559552] Lustre: Skipped 1 previous similar message [ 6365.665352] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6365.668932] Lustre: Skipped 1 previous similar message [ 6374.367194] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6374.637290] Lustre: server umount lustre-MDT0000 complete [ 6375.905142] LustreError: 172116:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6375.920848] LustreError: 172116:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6377.163838] LustreError: 172101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783447385 with bad export cookie 11042585386058579267 [ 6377.165419] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6377.169242] LustreError: 172101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6377.363095] Lustre: server umount lustre-MDT0001 complete [ 6384.422820] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 6386.215981] Key type lgssc unregistered [ 6386.460842] LNet: 227259:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6386.465134] LNetError: 227259:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6386.477450] LNet: Removed LNI 192.168.202.142@tcp [ 6386.983820] Key type .llcrypt unregistered [ 6386.985496] Key type ._llcrypt unregistered [ 6403.157207] Key type ._llcrypt registered [ 6403.159286] Key type .llcrypt registered [ 6403.248297] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 6404.074084] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6404.158341] alg: No test for adler32 (adler32-zlib) [ 6405.146679] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 6405.310328] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 6406.943193] Key type lgssc registered [ 6407.675791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6412.380617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6412.392221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6413.630205] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6416.049396] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6420.988047] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6421.207362] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6423.985242] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6428.159312] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6431.280635] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6435.188516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6435.411040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6438.340622] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6438.516932] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6442.273691] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6450.374408] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6451.360450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6451.368539] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6451.373296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.379702] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6451.379834] LustreError: 228672:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6454.177191] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6454.181785] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6454.183544] Lustre: Skipped 1 previous similar message [ 6454.752132] Lustre: lustre-OST0000: Not available for connect from 192.168.202.42@tcp (stopping) [ 6454.755406] Lustre: Skipped 1 previous similar message [ 6458.548733] Lustre: server umount lustre-OST0000 complete [ 6471.383099] LustreError: 228672:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6488.291617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6488.504694] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6491.792357] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6494.630319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.638703] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.638754] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6494.646673] Lustre: Skipped 1 previous similar message [ 6495.507517] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6496.398330] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6499.524727] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff89f00b5a4800.ost_server_uuid 50 [ 6500.422956] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff89f00b5a4800.ost_server_uuid in FULL state after 0 sec [ 6504.929154] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6504.932774] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6504.941859] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6504.944346] Lustre: Skipped 1 previous similar message [ 6508.450107] Lustre: server umount lustre-OST0000 complete [ 6513.632230] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6513.636072] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6513.640813] Lustre: Skipped 1 previous similar message [ 6513.643530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6516.555944] Lustre: server umount lustre-MDT0000 complete [ 6518.255156] LustreError: 228634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783447527 with bad export cookie 9195990585024705177 [ 6518.258795] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6518.260474] LustreError: 228634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6518.437104] Lustre: server umount lustre-MDT0001 complete [ 6523.837902] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 6525.453503] Key type lgssc unregistered [ 6525.621587] LNet: 233855:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6525.629240] LNetError: 233855:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6525.641449] LNet: Removed LNI 192.168.202.142@tcp [ 6526.117711] Key type .llcrypt unregistered [ 6526.119615] Key type ._llcrypt unregistered [ 6543.787290] Key type ._llcrypt registered [ 6543.789047] Key type .llcrypt registered [ 6543.837808] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 6544.533978] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6544.556182] alg: No test for adler32 (adler32-zlib) [ 6545.511837] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 6545.650281] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 6547.271226] Key type lgssc registered [ 6547.923885] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6553.728233] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6553.740659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6555.005434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6558.035717] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6563.278040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6563.446314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6565.994941] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6567.770933] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6571.521112] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6571.730349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6574.635569] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6578.641572] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6578.743440] Lustre: lustre-OST0001: new disk, initializing [ 6578.746434] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6578.751037] Lustre: 237865:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6578.817429] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6579.826676] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6579.921767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6579.929622] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6580.010801] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6581.878282] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6589.550176] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6593.048336] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6600.679549] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6600.681319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6600.685485] Lustre: Skipped 2 previous similar messages [ 6600.690882] Lustre: Skipped 3 previous similar messages [ 6605.793922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6605.801573] Lustre: Skipped 3 previous similar messages [ 6606.063804] Lustre: server umount lustre-MDT0000 complete [ 6607.938575] LustreError: 236694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783447616 with bad export cookie 15673604321608698595 [ 6607.940932] LustreError: 236298:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 6607.942264] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6607.946433] LustreError: 236694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6608.097827] Lustre: server umount lustre-MDT0001 complete [ 6620.184321] Lustre: server umount lustre-OST0000 complete [ 6632.634758] Lustre: server umount lustre-OST0001 complete [ 6643.981150] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 6650.806714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6651.063919] LustreError: 240972:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6651.075198] LustreError: 240972:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6651.112314] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6653.766509] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6656.481106] LustreError: 240973:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6659.042467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6659.304434] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6661.499041] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6663.008831] Lustre: 242113:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6666.191847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6669.411213] LustreError: 242466:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6669.420992] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 6670.442933] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6674.921502] LustreError: 242466:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6675.947858] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 6676.149513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6676.336372] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6676.341979] Lustre: Skipped 1 previous similar message [ 6680.055163] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6685.313810] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6687.215679] Lustre: 243982:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6695.394558] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6695.398858] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6695.405480] Lustre: Skipped 1 previous similar message [ 6695.408344] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6695.815626] Lustre: server umount mylustre-MDT0000 complete [ 6696.929405] LustreError: 240973:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6696.933959] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6696.944271] LustreError: 240973:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6696.955571] Lustre: Skipped 1 previous similar message [ 6698.147580] LustreError: 241716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783447706 with bad export cookie 15673604321608701430 [ 6698.149595] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6698.156149] LustreError: 241716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6698.383323] Lustre: server umount mylustre-MDT0001 complete [ 6710.333338] Lustre: server umount mylustre-OST0000 complete [ 6722.868335] Lustre: server umount mylustre-OST0001 complete [ 6735.474910] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 6745.180972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6745.591086] LustreError: 246288:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6745.599623] LustreError: 246288:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6745.671614] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6752.490954] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6765.545581] LustreError: 246289:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6765.568822] LustreError: 246289:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6767.438306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6773.302874] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6776.657281] Lustre: 247432:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6783.367850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6791.270897] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 6792.309268] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 6792.528887] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6797.820894] LustreError: 247787:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6797.840397] LustreError: 247787:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6802.957259] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6808.576852] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 6809.914830] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6818.319180] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6821.703704] Lustre: 249302:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6834.148865] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6834.156646] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6834.162283] Lustre: Skipped 4 previous similar messages [ 6834.165056] Lustre: Skipped 2 previous similar messages [ 6836.385584] Lustre: server umount tfs-MDT0000 complete [ 6840.270502] LustreError: 246264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783447849 with bad export cookie 15673604321608703383 [ 6840.296542] LustreError: 246264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6840.301218] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6840.880858] Lustre: server umount tfs-MDT0001 complete [ 6855.114683] Lustre: server umount tfs-OST0000 complete [ 6868.907689] Lustre: server umount tfs-OST0001 complete [ 6888.055355] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 6897.787760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6898.450259] LustreError: 251553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6898.478473] LustreError: 251553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 6898.609924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6898.618126] Lustre: Skipped 3 previous similar messages [ 6902.554207] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6910.108981] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6915.729737] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6918.408472] Lustre: 252696:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6924.065726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6930.545707] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6935.608895] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 6938.754866] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6944.147097] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 6944.157425] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 6945.273494] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 6952.256977] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6955.789822] Lustre: 254568:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6962.572506] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 14:12:49 (1783447969) [ 6977.506882] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6977.515063] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6977.534238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6977.545151] Lustre: Skipped 1 previous similar message [ 6983.914847] Lustre: server umount lustre-MDT0000 complete [ 6987.389443] LustreError: 251530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783447996 with bad export cookie 15673604321608705301 [ 6987.397199] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6987.403769] LustreError: 251530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6990.311482] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6990.323708] Lustre: Skipped 3 previous similar messages [ 6990.331236] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6990.338043] Lustre: Skipped 5 previous similar messages [ 6993.817568] Lustre: server umount lustre-MDT0001 complete [ 7003.385792] Lustre: server umount lustre-OST0000 complete [ 7013.312850] Lustre: server umount lustre-OST0001 complete [ 7019.671896] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 7028.119757] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 7061.789150] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7061.789150] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7061.789150] [ 7061.810384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7062.021647] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7062.038442] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7062.078767] Lustre: lustre-MDT0000: new disk, initializing [ 7062.115803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7062.119687] Lustre: Skipped 3 previous similar messages [ 7062.129444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7066.510655] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7075.199429] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7075.199429] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7075.199429] [ 7075.227880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7075.286358] Lustre: 257754:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7075.323411] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7075.326687] Lustre: Skipped 1 previous similar message [ 7075.383205] Lustre: lustre-MDT0001: new disk, initializing [ 7075.490142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7075.497934] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7079.401648] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7088.833405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7089.082355] Lustre: lustre-OST0000: new disk, initializing [ 7089.087414] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7089.096472] Lustre: 259335:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7091.125159] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7091.133753] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7091.254288] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7094.855562] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7104.226668] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7104.399870] Lustre: lustre-OST0001: new disk, initializing [ 7104.405739] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7104.411047] Lustre: 260205:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7105.887895] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7105.905359] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7105.986187] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7109.716953] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7121.982755] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 14:15:28 (1783448128) [ 7137.842221] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 14:15:45 (1783448145) [ 7152.102449] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7152.114204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7152.126766] Lustre: Skipped 2 previous similar messages [ 7152.140484] Lustre: Skipped 3 previous similar messages [ 7157.933255] Lustre: server umount lustre-OST0000 complete [ 7162.344533] LustreError: 259320:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7162.364882] LustreError: 259320:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 7168.208772] Lustre: server umount lustre-MDT0000 complete [ 7171.360936] LustreError: 258548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783448180 with bad export cookie 15673604321608707072 [ 7171.365561] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7171.369090] LustreError: 258548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7177.932320] Lustre: server umount lustre-MDT0001 complete [ 7188.363758] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 7198.979560] Lustre: server umount lustre-OST0001 complete [ 7205.708753] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 7219.452220] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 7249.780372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7249.987823] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7250.007897] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7250.088910] Lustre: lustre-MDT0000: new disk, initializing [ 7250.219132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7254.033881] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7262.642707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7262.744548] Lustre: 264758:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7262.967723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7266.502835] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7273.997095] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7278.509412] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7283.827962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7284.047478] Lustre: lustre-OST0000: new disk, initializing [ 7284.050578] Lustre: Skipped 1 previous similar message [ 7284.056769] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7284.064279] Lustre: Skipped 2 previous similar messages [ 7284.076559] Lustre: 266724:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7286.153798] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7286.157168] Lustre: Skipped 1 previous similar message [ 7286.171074] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7286.290397] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7289.008569] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7296.558308] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7301.603835] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7301.607547] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7301.612421] Lustre: Skipped 4 previous similar messages [ 7301.613587] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7301.613592] Lustre: Skipped 10 previous similar messages [ 7307.280368] Lustre: server umount lustre-OST0000 complete [ 7316.550434] Lustre: server umount lustre-MDT0000 complete [ 7320.237147] LustreError: 264751:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783448329 with bad export cookie 15673604321608708549 [ 7320.240822] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7320.247869] LustreError: 264751:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7320.736886] Lustre: server umount lustre-MDT0001 complete [ 7332.838769] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 7335.734785] Key type lgssc unregistered [ 7335.978627] LNet: 268484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7335.985945] LNetError: 268484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7337.045322] LNet: Removed LNI 192.168.202.142@tcp [ 7337.973459] Key type .llcrypt unregistered [ 7337.975811] Key type ._llcrypt unregistered [ 7365.196221] Key type ._llcrypt registered [ 7365.200551] Key type .llcrypt registered [ 7365.302721] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 7366.469220] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7366.626449] alg: No test for adler32 (adler32-zlib) [ 7367.741719] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 7367.930837] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 7369.591173] Key type lgssc registered [ 7370.608431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7377.780252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7377.812384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7379.274657] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7383.227293] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7389.664222] LustreError: 269949:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7389.689580] LustreError: 269949:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7391.109030] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7391.332949] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7395.745824] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7403.107297] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7408.385963] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7414.269192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7414.544604] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7420.546565] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7424.815692] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7427.348755] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7428.357736] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7430.112597] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7430.131328] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7430.134769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7430.162299] Lustre: Skipped 1 previous similar message [ 7435.235525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7435.238782] Lustre: Skipped 2 previous similar messages [ 7436.302967] Lustre: server umount lustre-OST0000 complete [ 7440.368745] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7440.374668] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7440.385192] Lustre: Skipped 1 previous similar message [ 7440.399484] Lustre: Skipped 1 previous similar message [ 7445.473701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7445.480770] Lustre: Skipped 1 previous similar message [ 7450.596762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7450.604282] Lustre: Skipped 1 previous similar message [ 7454.177630] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7454.353731] Lustre: server umount lustre-MDT0000 complete [ 7455.713110] LustreError: 270678:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7455.735170] LustreError: 270678:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7457.616809] LustreError: 272477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783448466 with bad export cookie 16885391693996392245 [ 7457.619385] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7457.624346] LustreError: 272477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7457.910790] Lustre: server umount lustre-MDT0001 complete [ 7467.508861] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 7469.887822] Key type lgssc unregistered [ 7470.163690] LNet: 273533:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7470.171539] LNetError: 273533:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7470.192687] LNet: Removed LNI 192.168.202.142@tcp [ 7471.130742] Key type .llcrypt unregistered [ 7471.132262] Key type ._llcrypt unregistered [ 7485.444328] Key type ._llcrypt registered [ 7485.452089] Key type .llcrypt registered [ 7485.593460] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 7486.604195] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7486.720831] alg: No test for adler32 (adler32-zlib) [ 7487.845843] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 7487.996647] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 7489.607187] Key type lgssc registered [ 7490.540144] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7497.678953] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7497.703755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7513.247531] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7518.367505] LustreError: 274719:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.142@tcp: failed processing log, type 4: rc = -110 [ 7544.991945] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7550.759895] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7557.199935] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7559.096378] Lustre: server umount lustre-OST0000 complete [ 7572.235460] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 7574.449559] Key type lgssc unregistered [ 7574.736909] LNet: 276349:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7574.745118] LNetError: 276349:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7574.763243] LNet: Removed LNI 192.168.202.142@tcp [ 7575.530162] Key type .llcrypt unregistered [ 7575.533790] Key type ._llcrypt unregistered [ 7606.649869] Key type ._llcrypt registered [ 7606.654628] Key type .llcrypt registered [ 7606.764532] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 7607.898691] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7607.946214] alg: No test for adler32 (adler32-zlib) [ 7609.025375] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 7609.196766] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 7610.863271] Key type lgssc registered [ 7611.836765] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7622.080698] loop0: detected capacity change from 0 to 204800000 [ 7622.226828] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7622.237694] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7622.520127] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7623.299290] loop0: detected capacity change from 0 to 204800000 [ 7623.467340] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7623.477379] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7623.761311] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7624.493530] loop0: detected capacity change from 0 to 204800000 [ 7624.617894] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7624.638595] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7624.762316] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7625.483700] loop0: detected capacity change from 0 to 204800000 [ 7625.616720] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7626.482491] loop0: detected capacity change from 0 to 204800000 [ 7626.511966] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7629.740204] loop0: detected capacity change from 0 to 204800000 [ 7629.760297] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7632.745577] loop0: detected capacity change from 0 to 204800000 [ 7632.779651] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7636.066725] loop0: detected capacity change from 0 to 204800000 [ 7636.084411] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7639.204308] loop0: detected capacity change from 0 to 204800000 [ 7639.243731] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7639.279075] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7639.334876] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7639.342109] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7639.349801] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7639.358528] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7639.379039] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7639.393179] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7639.397477] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7639.403125] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7639.409555] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7639.414972] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7639.422405] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7639.428378] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7639.434344] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7639.439769] Lustre: 279758:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7644.065610] Lustre: server umount MGS complete [ 7644.871292] loop0: detected capacity change from 0 to 204800000 [ 7644.972550] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7645.272187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7645.310785] Lustre: lustre-MDT0000: Aborting client recovery [ 7645.322042] LustreError: 280132:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 7646.142791] loop1: detected capacity change from 0 to 204800000 [ 7646.316947] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7646.433666] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7646.446787] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7646.458267] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7646.465807] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7646.471215] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7646.477713] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7646.483905] Lustre: 280189:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7646.497956] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7646.623924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7646.671994] Lustre: lustre-MDT0001: Aborting client recovery [ 7647.424510] loop2: detected capacity change from 0 to 204800000 [ 7647.521107] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7647.631227] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7647.639660] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7647.649458] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7647.665111] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7647.679191] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7647.689780] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7647.701097] Lustre: 280269:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7647.815538] Lustre: Found index 0 for lustre-OST0000, updating log [ 7647.853351] Lustre: 280289:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7647.931568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7647.951570] Lustre: lustre-OST0000: Aborting recovery [ 7648.606603] loop3: detected capacity change from 0 to 204800000 [ 7648.787616] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7648.855511] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7648.862938] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7648.873052] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7648.879293] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7648.886381] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7648.893492] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7648.900439] Lustre: 280356:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7648.912552] Lustre: Found index 1 for lustre-OST0001, updating log [ 7648.932707] Lustre: 280361:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7648.983772] Lustre: lustre-OST0001: Aborting recovery [ 7654.380935] LustreError: 280133:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 7655.401954] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7655.418259] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7655.432960] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7655.451578] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7656.426479] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7656.429947] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7661.026677] LustreError: 280100:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 409us: rc = -2 [ 7661.548228] LustreError: 280101:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 1555us: rc = -2 [ 7661.564804] LustreError: 280101:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 29 previous similar messages [ 7676.897276] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7676.914107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7679.765460] Lustre: server umount lustre-MDT0000 complete [ 7680.468876] LustreError: 280083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783448689 with bad export cookie 6132627157681552737 [ 7680.483967] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7682.017175] LustreError: 280097:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7682.017519] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7682.037777] LustreError: 280097:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7682.050245] Lustre: Skipped 3 previous similar messages [ 7682.058242] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7682.063471] Lustre: Skipped 4 previous similar messages [ 7685.023850] LustreError: 280099:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7685.031561] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7685.044072] Lustre: Skipped 1 previous similar message [ 7686.872818] Lustre: server umount lustre-MDT0001 complete [ 7693.912501] Lustre: server umount lustre-OST0000 complete [ 7709.152150] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7709.345341] Lustre: server umount lustre-OST0001 complete [ 7715.345646] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 14:25:22 (1783448722) [ 7738.521259] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 7746.286480] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 7774.494920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7774.691837] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7774.713893] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7774.783288] Lustre: lustre-MDT0000: new disk, initializing [ 7774.857144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7774.863373] Lustre: Skipped 1 previous similar message [ 7774.874378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7778.486703] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7786.212897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7786.329639] Lustre: 284433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7786.370614] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7786.372368] Lustre: Skipped 1 previous similar message [ 7786.427583] Lustre: lustre-MDT0001: new disk, initializing [ 7786.503603] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7786.533799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7786.542259] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7790.512463] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7797.606645] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7801.254915] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7805.745500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7805.910753] Lustre: lustre-OST0000: new disk, initializing [ 7805.912398] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7805.915630] Lustre: 286401:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7805.969114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7810.430352] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7812.141544] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7812.153765] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7812.229059] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7817.058188] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7821.156354] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7829.933857] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7837.415916] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7878.629375] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7878.639192] Lustre: Skipped 1 previous similar message [ 7878.645061] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7878.652402] Lustre: Skipped 1 previous similar message [ 7883.748480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7883.756043] Lustre: Skipped 1 previous similar message [ 7889.375330] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7889.511262] Lustre: server umount lustre-OST0000 complete [ 7893.985751] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7893.990531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7893.997687] Lustre: Skipped 2 previous similar messages [ 7894.012351] Lustre: Skipped 3 previous similar messages [ 7899.529546] Lustre: server umount lustre-MDT0000 complete [ 7902.425606] LustreError: 286400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783448911 with bad export cookie 6132627157681555460 [ 7902.432881] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7902.438947] LustreError: 286400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7902.693087] Lustre: server umount lustre-MDT0001 complete [ 7910.273571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7914.421792] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7918.154812] Lustre: server umount MGS complete [ 7926.357367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7926.750292] LustreError: 290354:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7926.762451] LustreError: 290354:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7926.778854] LustreError: 290367:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7926.822404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7930.146574] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7931.875120] LustreError: 290355:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7936.998199] LustreError: 290354:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7937.613695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7942.235728] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7948.538592] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7953.628183] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7959.170595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7959.571634] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7959.577079] Lustre: Skipped 1 previous similar message [ 7964.630412] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 7970.250386] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8002.531652] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8002.542119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8002.560060] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8002.566680] Lustre: Skipped 3 previous similar messages [ 8002.702414] Lustre: server umount lustre-OST0000 complete [ 8012.313193] Lustre: server umount lustre-MDT0000 complete [ 8012.770879] LustreError: 291396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8012.787800] LustreError: 291396:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8015.325988] LustreError: 293083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783449024 with bad export cookie 6132627157681557301 [ 8015.326288] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8015.350612] LustreError: 293083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8015.578182] Lustre: server umount lustre-MDT0001 complete [ 8023.610274] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 8025.892714] Key type lgssc unregistered [ 8026.199923] LNet: 294299:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8026.207110] LNetError: 294299:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8026.220298] LNet: Removed LNI 192.168.202.142@tcp [ 8026.932808] Key type .llcrypt unregistered [ 8026.937672] Key type ._llcrypt unregistered [ 8053.741772] Key type ._llcrypt registered [ 8053.744783] Key type .llcrypt registered [ 8053.846569] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 8065.012663] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 8065.774178] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8065.794355] alg: No test for adler32 (adler32-zlib) [ 8066.760360] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 8066.911533] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 8068.567176] Key type lgssc registered [ 8069.352849] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8094.314359] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8094.330720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8095.514598] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8095.533264] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8095.602671] Lustre: lustre-MDT0000: new disk, initializing [ 8095.661709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8095.673037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8098.900900] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8106.579099] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8106.686275] Lustre: 298234:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8106.750211] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8106.753934] Lustre: Skipped 1 previous similar message [ 8106.811644] Lustre: lustre-MDT0001: new disk, initializing [ 8106.951930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8106.998066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8107.009317] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8110.338379] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8116.648749] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8120.424747] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8125.741581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8125.935891] Lustre: lustre-OST0000: new disk, initializing [ 8125.939935] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8125.944426] Lustre: 300199:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8126.065069] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8131.432390] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8135.760378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8135.767348] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8135.884794] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8138.894784] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8145.058542] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8157.053377] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8166.030132] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8178.055578] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8223.204663] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8223.207120] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8223.209586] LustreError: Skipped 1 previous similar message [ 8223.211623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8223.221919] Lustre: Skipped 1 previous similar message [ 8224.361703] Lustre: server umount lustre-OST0000 complete [ 8228.320903] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8228.322030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8228.328717] Lustre: Skipped 1 previous similar message [ 8228.330840] Lustre: Skipped 1 previous similar message [ 8233.384573] Lustre: server umount lustre-MDT0000 complete [ 8235.835681] LustreError: 298227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783449244 with bad export cookie 9458388138419919457 [ 8235.840059] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8235.845532] LustreError: 298227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8236.143557] Lustre: server umount lustre-MDT0001 complete [ 8241.503918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8244.503957] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8247.671992] Lustre: server umount MGS complete [ 8256.063645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8256.361717] LustreError: 304329:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8256.375391] LustreError: 304329:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8256.376149] LustreError: 304342:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8256.443073] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8259.653793] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8261.605452] LustreError: 304330:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8265.797695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8266.139697] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8269.302966] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8274.045733] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8277.656302] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8282.357171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8282.602039] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8287.460616] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8293.421349] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8331.552143] Lustre: server umount lustre-OST0000 complete [ 8337.889523] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8337.889682] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8337.901215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8337.910743] Lustre: Skipped 1 previous similar message [ 8343.009638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8343.013042] Lustre: Skipped 1 previous similar message [ 8344.092798] Lustre: server umount lustre-MDT0000 complete [ 8346.438626] LustreError: 304310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783449355 with bad export cookie 9458388138419921410 [ 8346.439096] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8346.446906] LustreError: 304310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8356.940362] Lustre: server umount lustre-MDT0001 complete [ 8363.252201] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 8365.051290] Key type lgssc unregistered [ 8365.267805] LNet: 308297:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8365.271057] LNetError: 308297:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8365.286380] LNet: Removed LNI 192.168.202.142@tcp [ 8365.873360] Key type .llcrypt unregistered [ 8365.876278] Key type ._llcrypt unregistered [ 8377.681454] Key type ._llcrypt registered [ 8377.683301] Key type .llcrypt registered [ 8377.739514] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 8385.908433] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 8386.399608] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8386.479126] alg: No test for adler32 (adler32-zlib) [ 8387.381741] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 8387.503670] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 8389.111700] Key type lgssc registered [ 8389.767698] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8403.489383] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 8411.640214] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8411.652155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8412.774951] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8412.792394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8412.849377] Lustre: lustre-MDT0000: new disk, initializing [ 8412.899816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8412.912189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8415.612598] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8422.243246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8422.442220] Lustre: lustre-OST0000: new disk, initializing [ 8422.446533] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8422.450463] Lustre: Skipped 1 previous similar message [ 8422.455914] Lustre: 312320:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8422.501115] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8426.138418] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8430.632974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8430.638070] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8430.661279] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8879.009064] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8879.017949] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8884.194283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8884.659959] Lustre: server umount lustre-MDT0000 complete [ 8886.911075] LustreError: 311507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783449895 with bad export cookie 615216594111602006 [ 8886.913192] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8886.918687] LustreError: 311507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8887.013042] Lustre: server umount lustre-OST0000 complete [ 8902.346892] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 8908.697406] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 8929.780414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8929.946478] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8929.970527] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8930.027329] Lustre: lustre-MDT0000: new disk, initializing [ 8930.087343] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8930.101978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8932.084259] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8938.108741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8938.177186] Lustre: 316584:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8938.209276] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8938.211316] Lustre: Skipped 1 previous similar message [ 8938.265142] Lustre: lustre-MDT0001: new disk, initializing [ 8938.331237] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8938.348813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8938.359466] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8940.619850] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8946.071660] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8949.243920] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8953.135172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8953.286537] Lustre: lustre-OST0000: new disk, initializing [ 8953.289317] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8953.292894] Lustre: 318553:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8953.343954] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8956.275975] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8960.528464] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8960.535281] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8960.575601] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8961.164713] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8963.456801] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8963.559400] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8965.855284] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8966.009472] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8970.722799] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8970.723164] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8970.733733] LustreError: Skipped 1 previous similar message [ 8970.734581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8970.751108] Lustre: Skipped 1 previous similar message [ 8973.337560] Lustre: server umount lustre-OST0000 complete [ 8975.841257] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8975.851557] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8975.854220] Lustre: Skipped 1 previous similar message [ 8980.961029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8980.966475] Lustre: Skipped 1 previous similar message [ 8981.868734] Lustre: server umount lustre-MDT0000 complete [ 8984.160560] LustreError: 316577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783449992 with bad export cookie 615216594114542797 [ 8984.167797] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8984.173146] LustreError: 316577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8984.388393] Lustre: server umount lustre-MDT0001 complete [ 8986.466975] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 8987.456113] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 14:46:35 (1783449995) [ 8991.184334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8991.511873] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8994.009311] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 8996.835396] LustreError: 320863:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8996.844597] LustreError: 320863:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8999.698966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9002.351555] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9006.414294] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9009.401465] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9013.033563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9013.213803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9013.217202] Lustre: Skipped 1 previous similar message [ 9016.185740] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9020.065337] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9023.828830] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9023.961303] Lustre: lustre-OST0001: new disk, initializing [ 9023.963817] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9023.968577] Lustre: 323696:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9027.154520] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9032.769736] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9032.894463] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9033.754303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9033.760476] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9033.786546] LustreError: 324571:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9033.791926] LustreError: 324571:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9034.858314] LustreError: 324578:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9034.863613] LustreError: 324578:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9035.548811] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff89f007cb2000.ost_server_uuid 50 [ 9035.875323] LustreError: 324578:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9035.879402] LustreError: 324578:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9036.329647] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff89f007cb2000.ost_server_uuid in FULL state after 0 sec [ 9037.923782] LustreError: 324578:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9037.929024] LustreError: 324578:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9039.011110] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff89f007cb2000.ost_server_uuid 50 [ 9039.765566] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff89f007cb2000.ost_server_uuid in FULL state after 0 sec [ 9046.500196] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9054.179919] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9054.180750] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9054.184691] Lustre: Skipped 2 previous similar messages [ 9058.395082] Lustre: server umount lustre-OST0001 complete [ 9059.300690] LustreError: 322681:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9059.308136] LustreError: 322681:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9060.832416] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9060.837465] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9062.371912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9064.420025] LustreError: 323806:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9064.431050] LustreError: 323806:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9066.880304] Lustre: server umount lustre-OST0000 complete [ 9069.536837] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9069.542203] Lustre: Skipped 1 previous similar message [ 9074.658644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9074.665600] Lustre: Skipped 9 previous similar messages [ 9083.359150] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9083.434840] Lustre: server umount lustre-MDT0000 complete [ 9084.897947] LustreError: 320859:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9084.907238] LustreError: 320859:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9085.251598] LustreError: 322703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783450094 with bad export cookie 615216594114543672 [ 9085.255832] LustreError: 322703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9085.257276] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9085.402423] Lustre: server umount lustre-MDT0001 complete [ 9090.523182] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 9091.916472] Key type lgssc unregistered [ 9092.089595] LNet: 326378:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9092.092892] LNetError: 326378:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9092.104202] LNet: Removed LNI 192.168.202.142@tcp [ 9092.587124] Key type .llcrypt unregistered [ 9092.591558] Key type ._llcrypt unregistered [ 9105.999105] Key type ._llcrypt registered [ 9106.000619] Key type .llcrypt registered [ 9106.055673] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 9106.626376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9106.640075] alg: No test for adler32 (adler32-zlib) [ 9107.527453] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 9107.696282] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 9109.319142] Key type lgssc registered [ 9109.965119] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9113.546368] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9113.557983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9114.774202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9116.688824] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9120.725340] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9124.334899] LustreError: 327777:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9124.343445] LustreError: 327777:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9124.357508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9124.557531] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9126.383877] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9130.775910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9130.968115] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9133.470051] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9135.077871] LustreError: 329455:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9136.734872] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9139.546710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9139.659709] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9140.715768] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9140.727632] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9142.388232] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9146.150609] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9150.804218] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9150.899720] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9237.986153] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9237.987647] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9237.990270] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9237.997417] Lustre: Skipped 1 previous similar message [ 9240.796232] Lustre: server umount lustre-OST0001 complete [ 9243.107795] LustreError: 330587:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9243.129251] LustreError: 330587:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9247.716094] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9247.721162] LustreError: Skipped 1 previous similar message [ 9247.724058] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9247.730610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9247.733216] Lustre: Skipped 1 previous similar message [ 9248.226114] LustreError: 329979:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9248.236448] LustreError: 329979:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9249.458098] Lustre: server umount lustre-OST0000 complete [ 9252.832158] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9252.835936] LustreError: Skipped 1 previous similar message [ 9252.838353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9252.844018] Lustre: Skipped 1 previous similar message [ 9252.846985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9252.850040] Lustre: Skipped 2 previous similar messages [ 9257.659115] Lustre: server umount lustre-MDT0000 complete [ 9258.464161] LustreError: 327776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9258.472641] LustreError: 327776:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9259.324364] LustreError: 327759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783450268 with bad export cookie 17910434825399752954 [ 9259.325947] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9259.331465] LustreError: 327759:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9259.511907] Lustre: server umount lustre-MDT0001 complete [ 9264.593534] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 9266.050705] Key type lgssc unregistered [ 9266.230862] LNet: 332714:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9266.235437] LNetError: 332714:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9266.244910] LNet: Removed LNI 192.168.202.142@tcp [ 9266.776895] Key type .llcrypt unregistered [ 9266.779057] Key type ._llcrypt unregistered [ 9291.716404] Key type ._llcrypt registered [ 9291.717428] Key type .llcrypt registered [ 9291.785103] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 9292.471756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9292.506396] alg: No test for adler32 (adler32-zlib) [ 9293.441293] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 9293.575610] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 9295.199185] Key type lgssc registered [ 9295.802996] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9300.337315] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9300.352577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9301.607987] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9303.709781] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9308.027548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9308.201639] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9310.064533] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9311.637036] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9314.945685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9315.134927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9318.005890] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9318.181791] LustreError: 336483:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9318.190714] LustreError: 336483:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9322.181785] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9322.313467] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9325.107170] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9326.456414] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9326.460372] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9334.753351] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9344.339672] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9351.283827] Lustre: server umount lustre-MDT0000 complete [ 9352.163578] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9352.165150] LustreError: 334991:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9352.169197] Lustre: Skipped 3 previous similar messages [ 9352.177117] LustreError: 334991:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9353.078176] LustreError: 338002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783450361 with bad export cookie 12397873334586459113 [ 9353.078825] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9353.081615] LustreError: 338002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9355.330596] Lustre: server umount lustre-MDT0001 complete [ 9367.583632] Lustre: server umount lustre-OST0000 complete [ 9379.890706] Lustre: server umount lustre-OST0001 complete [ 9386.917283] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 9391.634905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9391.780518] LustreError: 340254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9391.788573] LustreError: 340254:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9391.806115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9393.692380] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9397.217206] LustreError: 340255:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9398.179717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9400.262357] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9401.658449] Lustre: 341392:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9404.933833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9405.174351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9405.177069] Lustre: Skipped 1 previous similar message [ 9407.831084] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9408.235664] LustreError: 341747:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9411.564819] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9414.667913] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9416.816140] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9416.817409] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9424.301321] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9425.958317] Lustre: 343265:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9427.425020] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9427.429424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9427.432925] Lustre: Skipped 3 previous similar messages [ 9427.437126] Lustre: Skipped 2 previous similar messages [ 9432.549412] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9432.551507] Lustre: Skipped 4 previous similar messages [ 9433.527601] Lustre: server umount lustre-MDT0000 complete [ 9435.357705] LustreError: 343268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783450444 with bad export cookie 12397873334586460814 [ 9435.359808] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9435.363630] LustreError: 343268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9435.514340] Lustre: server umount lustre-MDT0001 complete [ 9447.893585] Lustre: server umount lustre-OST0000 complete [ 9459.878539] Lustre: server umount lustre-OST0001 complete [ 9461.368784] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9462.038525] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9462.745284] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 14:54:30 (1783450470) [ 9472.355854] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 9476.391086] loop0: detected capacity change from 0 to 1099511627776 [ 9476.588093] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9476.595631] *** VALIDATE xfs *** [ 9476.601434] XFS (loop0): Mounting V5 Filesystem [ 9476.612791] XFS (loop0): Ending clean mount [ 9477.237142] loop1: detected capacity change from 0 to 18691697672192 [ 9506.047518] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [ 9510.264121] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 9525.905562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9526.045802] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9526.067937] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9526.109381] Lustre: lustre-MDT0000: new disk, initializing [ 9526.145986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9526.148288] Lustre: Skipped 1 previous similar message [ 9526.153560] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9528.086858] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9533.929496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9533.985687] Lustre: 348380:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9534.023980] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9534.026392] Lustre: Skipped 1 previous similar message [ 9534.072344] Lustre: lustre-MDT0001: new disk, initializing [ 9534.181272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9534.187113] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9536.293891] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9541.302510] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9543.799258] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9547.101751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9547.235409] Lustre: lustre-OST0000: new disk, initializing [ 9547.239311] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9547.244835] Lustre: 350347:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9548.997362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9549.001429] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9549.051324] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9550.114818] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9554.987455] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9556.825487] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9556.936020] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9558.754266] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9558.845906] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9564.642503] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9564.642615] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9564.643408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9564.645302] LustreError: Skipped 1 previous similar message [ 9564.649751] Lustre: Skipped 1 previous similar message [ 9565.719203] Lustre: server umount lustre-OST0000 complete [ 9569.761658] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9569.762479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9569.766581] Lustre: Skipped 1 previous similar message [ 9569.768276] Lustre: Skipped 1 previous similar message [ 9573.306109] Lustre: server umount lustre-MDT0000 complete [ 9574.880133] LustreError: 349188:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9574.885589] LustreError: 349188:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9574.958464] LustreError: 348372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783450583 with bad export cookie 12397873334586462361 [ 9574.962665] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9574.962942] LustreError: 348372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9575.105696] Lustre: server umount lustre-MDT0001 complete [ 9579.058753] XFS (loop0): Unmounting Filesystem [ 9581.108471] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 14:56:29 (1783450589) [ 9583.978879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9584.195552] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9584.202027] Lustre: Skipped 2 previous similar messages [ 9586.071812] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9590.215247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9592.402477] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9596.155726] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9598.621641] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9601.637740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9604.439672] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9607.385111] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9621.474197] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9621.476883] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9621.481433] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9621.484009] Lustre: Skipped 1 previous similar message [ 9622.788925] Lustre: server umount lustre-OST0000 complete [ 9626.567093] Lustre: server umount lustre-MDT0000 complete [ 9626.592814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9626.597865] LustreError: 355352:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9626.605266] LustreError: 355352:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9628.226519] LustreError: 352886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783450637 with bad export cookie 12397873334586463243 [ 9628.229563] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9628.231483] LustreError: 352886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9628.383996] Lustre: server umount lustre-MDT0001 complete [ 9632.828728] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [ 9634.076947] Key type lgssc unregistered [ 9634.233615] LNet: 356551:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9634.238307] LNetError: 356551:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9634.247964] LNet: Removed LNI 192.168.202.142@tcp [ 9634.610092] Key type .llcrypt unregistered [ 9634.611621] Key type ._llcrypt unregistered [ 9647.397188] Key type ._llcrypt registered [ 9647.398783] Key type .llcrypt registered [ 9647.445752] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [ 9647.727656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9647.733689] alg: No test for adler32 (adler32-zlib) [ 9648.612514] Lustre: Lustre: Build Version: 2.17.54_105_g2b8c50a [ 9648.718140] LNet: Added LNI 192.168.202.142@tcp [8/256/0/180] [ 9650.311658] Key type lgssc registered [ 9650.787139] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9653.881261] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9653.890777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9655.077956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9656.629777] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9660.242626] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9660.429067] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9662.149024] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9664.988905] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9667.193932] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9669.818166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9669.982639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9672.659585] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9675.962292] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9685.472903] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9685.478346] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9685.480512] Lustre: Skipped 1 previous similar message [ 9687.522127] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.42@tcp (stopping) [ 9690.592246] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9690.594432] Lustre: Skipped 1 previous similar message [ 9692.625452] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.42@tcp (stopping) [ 9697.750218] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.42@tcp (stopping) [ 9697.752437] Lustre: Skipped 2 previous similar messages [ 9699.295099] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9699.357820] Lustre: server umount lustre-MDT0001 complete [ 9700.834155] LustreError: 358094:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9700.841699] LustreError: 358094:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9702.868629] LustreError: 360720:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.42@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9703.608291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9703.653254] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9703.655670] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9703.671146] Lustre: 358084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9703.812644] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9705.416873] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [ 9709.028641] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9713.641128] LustreError: 361904:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9713.644064] LustreError: 361904:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9713.647959] LustreError: 358086:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9713.669668] LustreError: 361907:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.142@tcp: cfg command failed: rc = -17 [ 9713.674331] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9713.674331] [10010.076662] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.42@tcp (stopping) [10011.105395] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10011.105842] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10011.113673] Lustre: Skipped 1 previous similar message [10014.429550] Lustre: server umount lustre-MDT0001 complete [10015.195711] LustreError: 359333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.42@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10019.077822] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10019.146940] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10019.150737] Lustre: Found index 1 for lustre-MDT0001, updating log [10019.176255] Lustre: 358084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10019.336790] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10020.982287] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [10024.424568] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10029.042320] LustreError: 374006:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10029.045278] LustreError: 374006:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10029.050231] LustreError: 358086:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10029.053282] LustreError: 358086:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10029.093686] LustreError: 374013:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10029.098909] LustreError: 374013:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10029.104423] LustreError: 374013:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.142@tcp: cfg command failed: rc = -17 [10029.108366] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.142@tcp [10029.108366] [10326.498119] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10326.500521] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10326.503144] Lustre: Skipped 1 previous similar message [10326.508345] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10326.511161] Lustre: Skipped 3 previous similar messages [10338.271118] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10338.342157] Lustre: server umount lustre-MDT0001 complete [10341.856105] LustreError: 358113:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10341.861953] LustreError: 358113:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10342.578433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10342.636884] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10342.639312] Lustre: Found index 1 for lustre-MDT0001, updating log [10342.656277] Lustre: 358083:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10342.809692] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10344.356495] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [10348.005148] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10353.135167] LustreError: 386318:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10353.138099] LustreError: 386318:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10353.141737] LustreError: 358086:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10353.144232] LustreError: 358086:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10353.171552] LustreError: 386321:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10353.177869] LustreError: 386321:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.142@tcp: cfg command failed: rc = -114 [10353.182573] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10353.182573] [10660.321695] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10660.322516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10660.323091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10660.323096] Lustre: Skipped 7 previous similar messages [10660.325655] Lustre: Skipped 2 previous similar messages [10664.660075] Lustre: server umount lustre-MDT0000 complete [10665.441558] LustreError: 367244:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10665.446884] LustreError: 367244:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10665.812082] LustreError: 358120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783451674 with bad export cookie 10322405858728150805 [10665.815763] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10665.816270] LustreError: 358120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10670.560287] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10675.679932] LustreError: 359333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10675.684066] LustreError: 359333:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10680.287121] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10680.362967] Lustre: server umount lustre-MDT0001 complete [10681.653543] Lustre: server umount lustre-OST0000 complete [10684.589137] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [10687.715191] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [10700.009912] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:15:08 (1783451708) [10702.173446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10702.253360] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10702.262321] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10702.296882] Lustre: lustre-MDT0000: new disk, initializing [10702.315584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10702.320666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10703.616056] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [10707.761398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10707.806303] Lustre: 400679:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10707.832147] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10707.834387] Lustre: Skipped 1 previous similar message [10707.872982] Lustre: lustre-MDT0001: new disk, initializing [10707.906978] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10707.921356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10707.925711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10709.209227] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [10712.989871] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10714.873275] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10717.090529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10717.183408] Lustre: lustre-OST0000: new disk, initializing [10717.185212] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10717.187423] Lustre: 402647:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10717.215150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10719.012786] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_default_debug -1 all [10722.590281] Lustre: DEBUG MARKER: oleg242-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10725.365497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10725.369456] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10725.398836] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10733.535694] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10733.539132] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10733.544651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10733.547441] Lustre: Skipped 4 previous similar messages [10736.743556] Lustre: server umount lustre-MDT0000 complete [10738.007050] LustreError: 400674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783451746 with bad export cookie 10322405858728152744 [10738.010129] LustreError: 401486:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [10738.010586] LustreError: MGC192.168.202.142@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10738.010959] LustreError: 400674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10738.125474] Lustre: server umount lustre-MDT0001 complete [10749.783985] Lustre: server umount lustre-OST0000 complete [10759.125624] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing set_hostid [10762.176406] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing load_modules_local [10772.180504] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10434 sec ========= 15:16:20 (1783451780) [10772.714281] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:16:20 (1783451780) === [10773.895407] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:16:21 (1783451781) === [10787.242415] Lustre: DEBUG MARKER: oleg242-server.virtnet: executing unload_modules_local [10788.266424] Key type lgssc unregistered [10788.393355] LNet: 409001:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10788.395942] LNetError: 409001:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10788.406261] LNet: Removed LNI 192.168.202.142@tcp [10788.711962] Key type .llcrypt unregistered [10788.713066] Key type ._llcrypt unregistered