[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 489987100 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001015] APIC: Switch to symmetric I/O mode setup [ 0.003336] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.008526] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009026] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010014] pid_max: default: 32768 minimum: 301 [ 0.011000] LSM: Security Framework initializing [ 0.011000] Yama: becoming mindful. [ 0.011045] SELinux: Initializing. [ 0.012086] *** VALIDATE selinux *** [ 0.021166] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026128] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027164] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028118] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029137] *** VALIDATE tmpfs *** [ 0.031304] *** VALIDATE proc *** [ 0.032280] *** VALIDATE cgroup *** [ 0.033014] *** VALIDATE cgroup2 *** [ 0.034291] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035166] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037036] Spectre V2 : User space: Vulnerable [ 0.038012] Speculative Store Bypass: Vulnerable [ 0.041765] debug: unmapping init [mem 0xffffffffb4259000-0xffffffffb4260fff] [ 0.043954] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044744] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045028] ... version: 2 [ 0.046019] ... bit width: 48 [ 0.047012] ... generic registers: 4 [ 0.048012] ... value mask: 0000ffffffffffff [ 0.049086] ... max period: 00007fffffffffff [ 0.050019] ... fixed-purpose events: 3 [ 0.051015] ... event mask: 000000070000000f [ 0.053325] rcu: Hierarchical SRCU implementation. [ 0.055701] smp: Bringing up secondary CPUs ... [ 0.056604] x86: Booting SMP configuration: [ 0.057030] .... node #0, CPUs: #1 #2 #3 [ 0.061026] smp: Brought up 1 node, 4 CPUs [ 0.063020] smpboot: Max logical packages: 1 [ 0.064036] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.184557] node 0 deferred pages initialised in 118ms [ 0.187265] devtmpfs: initialized [ 0.188292] x86/mm: Memory block size: 128MB [ 0.191499] gcov: version magic: 0x41383552 [ 0.193380] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.194098] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.195374] pinctrl core: initialized pinctrl subsystem [ 0.196357] [ 0.197000] ************************************************************* [ 0.199022] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.202016] ** ** [ 0.205015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.208018] ** ** [ 0.210014] ** This means that this kernel is built to expose internal ** [ 0.213060] ** IOMMU data structures, which may compromise security on ** [ 0.216016] ** your system. ** [ 0.219018] ** ** [ 0.221016] ** If you see this message and you are not debugging the ** [ 0.224022] ** kernel, report this immediately to your vendor! ** [ 0.227017] ** ** [ 0.230022] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.234023] ************************************************************* [ 0.238091] NET: Registered protocol family 16 [ 0.240600] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.243081] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.246082] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.249095] cpuidle: using governor menu [ 0.251269] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.253655] PCI: Using configuration type 1 for base access [ 0.256159] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.264048] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.265046] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.268137] cryptd: max_cpu_qlen set to 1000 [ 0.270291] ACPI: Added _OSI(Module Device) [ 0.271009] ACPI: Added _OSI(Processor Device) [ 0.271955] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.273011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.276081] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.281013] ACPI: Interpreter enabled [ 0.282052] ACPI: PM: (supports S0 S3 S4 S5) [ 0.283032] ACPI: Using IOAPIC for interrupt routing [ 0.284236] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.287545] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.300062] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.302056] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.305036] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.309145] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.314675] acpiphp: Slot [2] registered [ 0.316159] acpiphp: Slot [5] registered [ 0.318228] acpiphp: Slot [6] registered [ 0.319151] acpiphp: Slot [7] registered [ 0.321148] acpiphp: Slot [8] registered [ 0.322152] acpiphp: Slot [9] registered [ 0.324162] acpiphp: Slot [10] registered [ 0.326194] acpiphp: Slot [3] registered [ 0.327152] acpiphp: Slot [4] registered [ 0.329150] acpiphp: Slot [11] registered [ 0.331151] acpiphp: Slot [12] registered [ 0.332158] acpiphp: Slot [13] registered [ 0.334138] acpiphp: Slot [14] registered [ 0.336156] acpiphp: Slot [15] registered [ 0.337134] acpiphp: Slot [16] registered [ 0.339139] acpiphp: Slot [17] registered [ 0.341181] acpiphp: Slot [18] registered [ 0.342234] acpiphp: Slot [19] registered [ 0.344137] acpiphp: Slot [20] registered [ 0.346167] acpiphp: Slot [21] registered [ 0.347130] acpiphp: Slot [22] registered [ 0.349132] acpiphp: Slot [23] registered [ 0.350126] acpiphp: Slot [24] registered [ 0.352148] acpiphp: Slot [25] registered [ 0.354142] acpiphp: Slot [26] registered [ 0.355121] acpiphp: Slot [27] registered [ 0.356219] acpiphp: Slot [28] registered [ 0.357094] acpiphp: Slot [29] registered [ 0.358128] acpiphp: Slot [30] registered [ 0.360135] acpiphp: Slot [31] registered [ 0.361189] PCI host bridge to bus 0000:00 [ 0.363026] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.365033] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.368034] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.371027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.373029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.376036] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.378221] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.381509] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.384468] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.394647] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.399482] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.401021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.402015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.404018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.405605] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.407712] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.410071] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.412569] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.416015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.425016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.430020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.435795] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.441024] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.448023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.466024] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.475648] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.483020] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.488024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.502030] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.510859] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.518023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.523023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.537024] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.545607] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.551019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.559023] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.578019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.584480] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.591022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.598025] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.611020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.618819] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.623024] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.629022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.645025] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.654283] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.656316] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.658341] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.659295] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.661284] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.666151] iommu: Default domain type: Passthrough [ 0.667471] SCSI subsystem initialized [ 0.668113] ACPI: bus type USB registered [ 0.670138] usbcore: registered new interface driver usbfs [ 0.671161] usbcore: registered new interface driver hub [ 0.673104] usbcore: registered new device driver usb [ 0.674150] pps_core: LinuxPPS API ver. 1 registered [ 0.675008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.677076] PTP clock support registered [ 0.679049] EDAC MC: Ver: 3.0.0 [ 0.680431] PCI: Using ACPI for IRQ routing [ 0.681472] NetLabel: Initializing [ 0.682010] NetLabel: domain hash size = 128 [ 0.682852] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.684077] NetLabel: unlabeled traffic allowed by default [ 0.686112] vgaarb: loaded [ 0.686569] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.688018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.695644] clocksource: Switched to clocksource kvm-clock [ 0.790262] VFS: Disk quotas dquot_6.6.0 [ 0.791600] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.793738] *** VALIDATE ramfs *** [ 0.794737] *** VALIDATE hugetlbfs *** [ 0.795933] pnp: PnP ACPI init [ 0.797916] pnp: PnP ACPI: found 6 devices [ 0.814058] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.816666] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.818331] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.819853] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.821572] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.823347] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.825415] NET: Registered protocol family 2 [ 0.827436] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.831711] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.834473] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.839039] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.841959] TCP: Hash tables configured (established 65536 bind 65536) [ 0.844255] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.846684] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.848733] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.851421] NET: Registered protocol family 1 [ 0.854771] RPC: Registered named UNIX socket transport module. [ 0.856148] RPC: Registered udp transport module. [ 0.857519] RPC: Registered tcp transport module. [ 0.858663] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.860553] NET: Registered protocol family 44 [ 0.862509] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.864971] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.867287] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.869711] PCI: CLS 0 bytes, default 64 [ 0.871154] Unpacking initramfs... [ 2.210149] debug: unmapping init [mem 0xffff8f9a7cc54000-0xffff8f9a7ffbffff] [ 2.213452] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.215062] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.216937] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.706265] Initialise system trusted keyrings [ 2.707479] Key type blacklist registered [ 2.708894] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.719435] zbud: loaded [ 2.722501] *** VALIDATE nfs *** [ 2.723576] *** VALIDATE nfs4 *** [ 2.724705] pstore: using deflate compression [ 2.727647] Platform Keyring initialized [ 2.834659] NET: Registered protocol family 38 [ 2.836911] Key type asymmetric registered [ 2.838923] Asymmetric key parser 'x509' registered [ 2.840963] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.843990] io scheduler mq-deadline registered [ 2.845747] io scheduler kyber registered [ 2.847401] io scheduler bfq registered [ 2.849316] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.852074] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.854690] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.857023] ACPI: Power Button [PWRF] [ 2.862493] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.868986] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.882115] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.890516] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.912220] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.940994] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.967795] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.972173] Non-volatile memory driver v1.3 [ 2.973563] Linux agpgart interface v0.103 [ 3.006665] virtio_blk virtio1: [vda] 136528 512-byte logical blocks (69.9 MB/66.7 MiB) [ 3.009676] vda: detected capacity change from 0 to 69902336 [ 3.029094] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.032070] vdb: detected capacity change from 0 to 1073741824 [ 3.048335] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.051371] vdc: detected capacity change from 0 to 2621440000 [ 3.067978] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.070808] vdd: detected capacity change from 0 to 2621440000 [ 3.086684] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.089653] vde: detected capacity change from 0 to 4294967296 [ 3.103125] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.105749] vdf: detected capacity change from 0 to 4294967296 [ 3.112313] libphy: Fixed MDIO Bus: probed [ 3.126842] usbcore: registered new interface driver usbserial_generic [ 3.128858] usbserial: USB Serial support registered for generic [ 3.130746] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.134384] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.136595] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.138875] mousedev: PS/2 mouse device common for all mice [ 3.142549] rtc_cmos 00:05: RTC can wake from S4 [ 3.144818] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.146815] rtc_cmos 00:05: registered as rtc0 [ 3.150471] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.150890] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.154588] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.156867] intel_pstate: CPU model not supported [ 3.162850] hid: raw HID events driver (C) Jiri Kosina [ 3.164841] usbcore: registered new interface driver usbhid [ 3.166645] usbhid: USB HID core driver [ 3.168259] drop_monitor: Initializing network drop monitor service [ 3.170121] Initializing XFRM netlink socket [ 3.172154] NET: Registered protocol family 10 [ 3.175305] Segment Routing with IPv6 [ 3.176537] NET: Registered protocol family 17 [ 3.178465] mpls_gso: MPLS GSO support [ 3.183314] RAS: Correctable Errors collector initialized. [ 3.185388] AVX version of gcm_enc/dec engaged. [ 3.187099] AES CTR mode by8 optimization enabled [ 3.260578] sched_clock: Marking stable (3260550307, 0)->(4155000115, -894449808) [ 3.264772] registered taskstats version 1 [ 3.268083] Loading compiled-in X.509 certificates [ 3.269972] zswap: loaded using pool lzo/zbud [ 3.298356] Key type big_key registered [ 3.310150] Key type encrypted registered [ 3.312092] ima: No TPM chip found, activating TPM-bypass! [ 3.314024] ima: Allocated hash algorithm: sha1 [ 3.315660] ima: No architecture policies found [ 3.317134] evm: Initialising EVM extended attributes: [ 3.318683] evm: security.selinux [ 3.319871] evm: security.ima [ 3.320882] evm: security.capability [ 3.321940] evm: HMAC attrs: 0x1 [ 3.324717] rtc_cmos 00:05: setting system clock to 2026-06-12 01:34:38 UTC (1781228078) [ 3.330894] debug: unmapping init [mem 0xffffffffb5203000-0xffffffffb53fffff] [ 3.333518] debug: unmapping init [mem 0xffffffffb3f82000-0xffffffffb4258fff] [ 3.342097] Write protecting the kernel read-only data: 28672k [ 3.345810] debug: unmapping init [mem 0xffffffffb2603000-0xffffffffb27fffff] [ 3.348430] debug: unmapping init [mem 0xffffffffb2f14000-0xffffffffb2ffffff] [ 3.386436] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.394029] systemd[1]: Detected virtualization kvm. [ 3.395726] systemd[1]: Detected architecture x86-64. [ 3.397296] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.422734] systemd[1]: No hostname configured. [ 3.424307] systemd[1]: Set hostname to . [ 3.426267] random: systemd: uninitialized urandom read (16 bytes read) [ 3.428668] systemd[1]: Initializing machine ID from random generator. [ 3.485905] random: ln: uninitialized urandom read (6 bytes read) [ 3.568738] random: systemd: uninitialized urandom read (16 bytes read) [ 3.571912] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.575598] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.578791] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Reached target Paths. Starting Create Volatile Files and Directories... [ OK ] Reached target Local Encrypted Volumes. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.253413] device-mapper: uevent: version 1.0.3 [ 4.255330] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.050762] random: fast init done [ 5.052966] virtio_net virtio0 ens2: renamed from eth0 [ 5.091898] scsi host0: ata_piix [ 5.174472] scsi host1: ata_piix [ 5.182801] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.184897] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.601861] dracut-initqueue[586]: RTNETLINK answers: File exists [ 9.827968] random: crng init done [ 9.829354] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.346719] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.563436] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.850932] SELinux: Disabled at runtime. [ 11.913504] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.922437] systemd[1]: Detected virtualization kvm. [ 11.924313] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.420652] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.423821] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.429790] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.435400] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.438811] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.445789] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.449940] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on Process Core Dump Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Reached target Slices. [ 12.613350] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.894151] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.234062] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.248524] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.318916] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.357365] EDAC sbridge: Ver: 1.1.2 [ 14.973936] Key type dns_resolver registered [ 15.273504] NFS: Registering the id_resolver key type [ 15.275616] Key type id_resolver registered [ 15.277222] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg102-server login: [ 50.811145] libcfs: loading out-of-tree module taints kernel. [ 50.847486] Key type ._llcrypt registered [ 50.849327] Key type .llcrypt registered [ 50.948088] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 69.903449] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 71.606601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 71.622376] alg: No test for adler32 (adler32-zlib) [ 73.162366] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 73.965566] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 75.831884] Key type lgssc registered [ 77.917645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 100.543600] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 135.338154] hrtimer: interrupt took 4132337 ns [ 150.498838] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 163.050856] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 163.076807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 164.343783] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 164.373607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 164.468530] Lustre: lustre-MDT0000: new disk, initializing [ 164.554376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 164.576608] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 169.459375] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.934656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 185.128980] Lustre: 6501:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 185.151309] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 185.158684] Lustre: Skipped 1 previous similar message [ 185.225378] Lustre: lustre-MDT0001: new disk, initializing [ 185.273949] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 185.313353] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 185.320993] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 190.008637] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 195.152347] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 205.464621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 205.690862] Lustre: lustre-OST0000: new disk, initializing [ 205.694636] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 205.699365] Lustre: 8405:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 205.754297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 208.532462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 208.546799] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 208.590289] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 211.763913] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 228.233285] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 228.435243] Lustre: lustre-OST0001: new disk, initializing [ 228.446539] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 228.454645] Lustre: 9461:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 228.573399] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 236.163558] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 237.690777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 237.702448] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 237.807522] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 248.902569] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 255.060757] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 262.639587] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing check_logdir /tmp/testlogs/ [ 268.357836] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing yml_node [ 273.786093] Lustre: DEBUG MARKER: Client: 2.17.53.64 [ 276.933205] Lustre: DEBUG MARKER: MDS: 2.17.53.64 [ 280.271595] Lustre: DEBUG MARKER: OSS: 2.17.53.64 [ 282.040351] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jun 11 21:39:15 EDT 2026 [ 298.995525] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 300.220676] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 318.433630] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 318.447357] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 318.460678] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 319.969424] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 319.970099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 319.984313] Lustre: Skipped 1 previous similar message [ 319.990563] Lustre: Skipped 1 previous similar message [ 325.900601] Lustre: server umount lustre-MDT0000 complete [ 330.209666] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.227430] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 330.326052] LustreError: 6492:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228405 with bad export cookie 7460172530166496558 [ 330.337680] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 330.339912] LustreError: 6492:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 335.329201] LustreError: 9055:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 335.332735] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 335.362238] LustreError: 9055:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 335.386243] Lustre: Skipped 1 previous similar message [ 335.393486] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 335.396424] Lustre: Skipped 3 previous similar messages [ 336.868660] Lustre: server umount lustre-MDT0001 complete [ 341.127575] Lustre: server umount lustre-OST0000 complete [ 344.862339] Lustre: server umount lustre-OST0001 complete [ 352.904334] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 363.547632] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 403.727802] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 404.047636] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 404.095969] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 404.186035] Lustre: lustre-MDT0000: new disk, initializing [ 404.293401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 404.323790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 409.508929] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 421.022948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 421.176615] Lustre: 14976:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 421.243405] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 421.246487] Lustre: Skipped 1 previous similar message [ 421.321927] Lustre: lustre-MDT0001: new disk, initializing [ 421.471425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 421.524208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 421.542895] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 426.613874] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 434.896790] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 438.755873] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 445.857632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 446.173842] Lustre: lustre-OST0000: new disk, initializing [ 446.179667] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 446.193630] Lustre: 16720:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 446.331553] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 447.471662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 447.486780] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 447.682759] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 453.017187] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 461.288577] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 466.247171] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 466.765369] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 472.867733] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 473.204652] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 478.179947] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 478.199466] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 478.237695] Lustre: Skipped 1 previous similar message [ 478.257966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 482.235376] Lustre: server umount lustre-OST0000 complete [ 493.029459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 493.042491] LustreError: Skipped 1 previous similar message [ 493.049483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 493.064664] Lustre: Skipped 1 previous similar message [ 493.069376] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 493.072330] Lustre: Skipped 1 previous similar message [ 495.892994] Lustre: server umount lustre-MDT0000 complete [ 498.146252] LustreError: 14985:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 498.195894] LustreError: 14985:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 500.689609] LustreError: 14969:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228575 with bad export cookie 7460172530166498091 [ 500.691544] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 500.698825] LustreError: 14969:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 501.414820] Lustre: server umount lustre-MDT0001 complete [ 509.079525] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 21:43:02 (1781228582) [ 516.864488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.338403] LustreError: 18837:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.471293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 521.745554] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 522.733115] LustreError: 18838:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 530.269258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 534.631954] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 539.803931] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 543.073707] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 549.308925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 555.937235] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 561.160187] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 570.061749] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 576.992612] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 576.995988] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 577.025065] Lustre: Skipped 1 previous similar message [ 577.039836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 577.053089] Lustre: Skipped 2 previous similar messages [ 582.568257] Lustre: server umount lustre-MDT0000 complete [ 583.650804] LustreError: 18838:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 583.686585] LustreError: 18838:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 587.179427] LustreError: 18818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228662 with bad export cookie 7460172530166498966 [ 587.184945] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 587.192041] LustreError: 18818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 587.648944] Lustre: server umount lustre-MDT0001 complete [ 602.377721] Lustre: server umount lustre-OST0000 complete [ 630.341791] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 641.541141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 642.065943] LustreError: 22860:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 642.105100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 642.110204] Lustre: Skipped 2 previous similar messages [ 646.396580] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 656.606136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 661.828517] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 665.183500] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 672.395950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 679.147982] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 687.767283] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 687.906147] Lustre: lustre-OST0001: new disk, initializing [ 687.908244] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 687.912093] Lustre: 25129:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 689.417699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 689.433024] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 689.581297] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 693.699202] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 704.331482] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 707.994906] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 723.938304] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 723.955831] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 723.970543] Lustre: Skipped 2 previous similar messages [ 723.975715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 723.979965] Lustre: Skipped 3 previous similar messages [ 727.268609] Lustre: server umount lustre-MDT0000 complete [ 730.603461] LustreError: 22855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 730.652170] LustreError: 22855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 730.858332] LustreError: 22841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228806 with bad export cookie 7460172530166499820 [ 730.867255] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 730.873344] LustreError: 22841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 731.291697] Lustre: server umount lustre-MDT0001 complete [ 744.087536] Lustre: server umount lustre-OST0000 complete [ 758.664551] Lustre: server umount lustre-OST0001 complete [ 775.499905] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 785.194466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 786.077243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 786.079459] Lustre: Skipped 3 previous similar messages [ 789.775907] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 796.642829] LustreError: 28113:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 796.673696] LustreError: 28113:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 798.797153] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 804.717473] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 808.129460] Lustre: 29226:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 816.090114] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 822.619613] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 832.386601] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 840.355735] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 850.211084] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 854.280711] Lustre: 31074:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 860.639910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 860.648315] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 860.662431] Lustre: Skipped 3 previous similar messages [ 860.672879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 860.680358] Lustre: Skipped 4 previous similar messages [ 866.784867] Lustre: server umount lustre-MDT0000 complete [ 870.809643] LustreError: 30400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228945 with bad export cookie 7460172530166501115 [ 870.815198] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 870.818876] LustreError: 30400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 871.313850] Lustre: server umount lustre-MDT0001 complete [ 886.393185] Lustre: server umount lustre-OST0000 complete [ 890.352172] Lustre: 3648:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781228949/real 1781228949] req@ffff8f9afd0570c0 x1867753087864448/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781228965 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 890.827771] Lustre: server umount lustre-OST0001 complete [ 899.751968] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 21:49:33 (1781228973) [ 931.252234] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 941.410843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 941.975953] LustreError: 33858:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 942.024381] LustreError: 33858:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 946.364514] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 954.679381] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 959.872771] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 963.022871] Lustre: 34973:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 971.394639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 979.525727] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 990.536210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 998.150254] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1007.444551] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1011.666183] Lustre: 36822:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1016.814861] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1016.827074] Lustre: Skipped 4 previous similar messages [ 1016.835213] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1016.842433] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1016.851617] Lustre: Skipped 6 previous similar messages [ 1022.710443] Lustre: server umount lustre-MDT0000 complete [ 1026.725448] LustreError: 34975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229101 with bad export cookie 7460172530166502221 [ 1026.738653] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1026.743984] LustreError: 34975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1027.247371] Lustre: server umount lustre-MDT0001 complete [ 1037.379668] Lustre: server umount lustre-OST0000 complete [ 1041.136767] Lustre: server umount lustre-OST0001 complete [ 1050.763542] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 21:52:04 (1781229124) [ 1068.109864] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 1078.105512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1078.865891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1078.878239] Lustre: Skipped 7 previous similar messages [ 1083.555259] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1091.970782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1097.023857] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1099.736661] Lustre: 40058:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1107.139370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1114.350652] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1122.827106] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1130.600096] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1140.036228] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1145.314063] Lustre: 41904:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1146.494920] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1146.504310] Lustre: Skipped 1 previous similar message [ 1153.491788] Lustre: server umount lustre-MDT0000 complete [ 1154.020945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1154.041088] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1154.059731] Lustre: Skipped 5 previous similar messages [ 1157.735894] LustreError: 40061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229232 with bad export cookie 7460172530166503327 [ 1157.750432] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1157.755584] LustreError: 40061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1159.531337] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.2@tcp (stopping) [ 1159.535883] Lustre: Skipped 4 previous similar messages [ 1164.604707] Lustre: server umount lustre-MDT0001 complete [ 1174.420375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1182.689437] LustreError: 3647:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8f9afcef61c0 x1867753087968640/t0(0) o250->MGC192.168.201.102@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1187.854770] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1192.373553] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1196.894429] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1202.425183] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1202.677574] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1202.695298] Lustre: Skipped 1 previous similar message [ 1207.944173] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1209.706061] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1214.419729] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1215.682904] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1223.141637] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1227.820500] Lustre: server umount lustre-MDT0000 complete [ 1228.274747] LustreError: 42779:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1228.295205] LustreError: 42779:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 38 previous similar messages [ 1231.341309] LustreError: 38925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229306 with bad export cookie 7460172530166504461 [ 1231.348461] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1231.355614] LustreError: 38925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1231.744781] Lustre: server umount lustre-MDT0001 complete [ 1245.910465] Lustre: server umount lustre-OST0000 complete [ 1249.253662] Lustre: 3650:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781229308/real 1781229308] req@ffff8f99c6535a40 x1867753088001408/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781229324 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1250.170897] Lustre: server umount lustre-OST0001 complete [ 1259.846346] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:55:33 (1781229333) [ 1273.807851] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 1283.116483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1287.815458] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1295.981280] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1300.403977] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1303.182984] Lustre: 47388:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1310.455144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1317.285591] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1327.449378] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1335.735525] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1345.351667] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1352.082642] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1375.662478] Lustre: server umount lustre-MDT0000 complete [ 1379.001041] LustreError: 46252:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229454 with bad export cookie 7460172530166505273 [ 1379.013863] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1379.016365] LustreError: 46252:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1379.436891] Lustre: server umount lustre-MDT0001 complete [ 1393.795723] Lustre: server umount lustre-OST0000 complete [ 1396.704092] Lustre: 3651:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781229455/real 1781229455] req@ffff8f9afb13c3c0 x1867753088065024/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781229471 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1397.536607] Lustre: server umount lustre-OST0001 complete [ 1407.567989] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:58:01 (1781229481) [ 1409.822489] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1411.299482] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:58:05 (1781229485) [ 1421.865737] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 1432.219886] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 1457.606508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1457.959872] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1458.052650] Lustre: lustre-MDT0000: new disk, initializing [ 1458.148777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1463.026537] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1470.999105] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1477.448806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1477.685257] Lustre: lustre-OST0000: new disk, initializing [ 1477.689163] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1477.695298] Lustre: Skipped 1 previous similar message [ 1477.699397] Lustre: 53306:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1479.079033] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1479.086835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1479.131888] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1484.024778] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1492.710192] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1494.719328] Lustre: DEBUG MARKER: create test files [ 1525.221245] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1525.228299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1525.240617] Lustre: Skipped 15 previous similar messages [ 1525.247225] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1525.252231] Lustre: Skipped 16 previous similar messages [ 1527.114766] Lustre: server umount lustre-OST0000 complete [ 1530.982348] Lustre: server umount lustre-MDT0000 complete [ 1543.060340] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 1546.294967] Key type lgssc unregistered [ 1546.649152] LNet: 54804:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1546.659935] LNetError: 54804:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1546.680686] LNet: Removed LNI 192.168.201.102@tcp [ 1548.111349] Key type .llcrypt unregistered [ 1548.115370] Key type ._llcrypt unregistered [ 1580.016742] Key type ._llcrypt registered [ 1580.018733] Key type .llcrypt registered [ 1580.133927] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 1582.120493] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1582.191700] alg: No test for adler32 (adler32-zlib) [ 1583.401443] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 1583.842249] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 1585.703173] Key type lgssc registered [ 1587.823120] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1597.280288] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1597.316445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1599.090781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1603.681049] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1609.272928] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1615.763275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1616.202671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1623.021832] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1624.366127] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1624.505990] Lustre: 57423:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1624.524043] Lustre: 57423:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1624.535048] Lustre: 57423:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1624.549474] Lustre: 57423:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1624.559886] Lustre: 57423:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1624.581102] Lustre: 57423:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1629.405949] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1634.778528] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1639.384394] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1647.995931] Lustre: server umount lustre-OST0000 complete [ 1662.251351] Lustre: server umount lustre-MDT0000 complete [ 1676.756471] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 1679.771637] Key type lgssc unregistered [ 1680.239827] LNet: 58904:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1680.251064] LNetError: 58904:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1681.318430] LNet: Removed LNI 192.168.201.102@tcp [ 1682.262473] Key type .llcrypt unregistered [ 1682.264861] Key type ._llcrypt unregistered [ 1713.328950] Key type ._llcrypt registered [ 1713.330683] Key type .llcrypt registered [ 1713.407967] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 1714.922753] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1714.961889] alg: No test for adler32 (adler32-zlib) [ 1716.084695] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 1716.317887] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 1718.008703] Key type lgssc registered [ 1719.666887] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1729.264800] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1729.295813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1731.032502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1737.229475] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1743.505411] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1751.625790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1752.136351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1756.221986] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1756.258375] Lustre: 61608:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1756.266669] Lustre: 61608:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1756.272436] Lustre: 61608:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1756.277326] Lustre: 61608:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1756.285214] Lustre: 61608:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1756.291581] Lustre: 61608:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1758.822898] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1765.391809] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1770.737677] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1781.744540] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1781.758768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1781.778615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1786.850091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1787.745681] Lustre: server umount lustre-OST0000 complete [ 1792.816616] Lustre: server umount lustre-MDT0000 complete [ 1805.896845] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 1809.005345] Key type lgssc unregistered [ 1809.339986] LNet: 63036:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1809.346646] LNetError: 63036:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1809.374071] LNet: Removed LNI 192.168.201.102@tcp [ 1810.208483] Key type .llcrypt unregistered [ 1810.210180] Key type ._llcrypt unregistered [ 1830.854482] Key type ._llcrypt registered [ 1830.856231] Key type .llcrypt registered [ 1831.026983] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 1847.672621] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 1849.028207] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1849.121344] alg: No test for adler32 (adler32-zlib) [ 1850.226806] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 1850.668430] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 1852.436049] Key type lgssc registered [ 1853.682615] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1893.794253] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1893.814919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1895.096993] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1895.117795] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1895.223641] Lustre: lustre-MDT0000: new disk, initializing [ 1895.430651] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1895.483424] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1901.041648] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1913.561926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1913.688530] Lustre: 66157:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1913.751144] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1913.763422] Lustre: Skipped 1 previous similar message [ 1913.845391] Lustre: lustre-MDT0001: new disk, initializing [ 1913.976947] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1914.020766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1914.041376] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1918.438507] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1926.880377] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1930.494823] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1938.828182] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1939.295593] Lustre: lustre-OST0000: new disk, initializing [ 1939.297896] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1939.308923] Lustre: 67898:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1939.442508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1945.764140] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 1948.270426] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1948.277197] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1948.385936] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1953.368738] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1957.250234] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1957.424737] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1961.792430] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1962.042524] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1967.020991] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1967.042082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1967.050711] Lustre: Skipped 1 previous similar message [ 1967.069337] Lustre: Skipped 1 previous similar message [ 1972.205280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1972.215601] Lustre: Skipped 1 previous similar message [ 1977.317500] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1977.329715] Lustre: Skipped 1 previous similar message [ 1978.337374] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1978.566364] Lustre: server umount lustre-OST0000 complete [ 1982.433224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1982.447718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1982.459468] Lustre: Skipped 1 previous similar message [ 1982.482034] Lustre: Skipped 1 previous similar message [ 1987.556608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1987.566639] Lustre: Skipped 1 previous similar message [ 1988.766084] Lustre: server umount lustre-MDT0000 complete [ 1992.645027] LustreError: 67897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230067 with bad export cookie 6741736566685172594 [ 1992.646127] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1992.652985] LustreError: 67897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1993.022799] Lustre: server umount lustre-MDT0001 complete [ 2003.195515] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:07:56 (1781230076) [ 2014.648475] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 2054.976794] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 2066.417058] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 2103.518239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2103.804607] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2103.833168] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2103.888558] Lustre: lustre-MDT0000: new disk, initializing [ 2103.958635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2103.998316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2108.239754] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2119.129711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2119.248474] Lustre: 73729:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2119.302149] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2119.308987] Lustre: Skipped 1 previous similar message [ 2119.382454] Lustre: lustre-MDT0001: new disk, initializing [ 2119.481190] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2119.518541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2119.541001] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2123.799520] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2132.361189] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2136.439437] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2142.964398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2143.221325] Lustre: lustre-OST0000: new disk, initializing [ 2143.224901] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2143.232638] Lustre: 75470:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2143.388284] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2149.715305] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2153.102311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2153.113446] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2153.214068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2157.412170] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2161.957208] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2162.203956] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2167.318865] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2167.512444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2173.921994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2173.923289] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2173.936668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2176.369521] Lustre: server umount lustre-OST0000 complete [ 2181.089465] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2181.103651] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2181.115712] Lustre: Skipped 1 previous similar message [ 2187.086518] Lustre: server umount lustre-MDT0000 complete [ 2190.856735] LustreError: 75469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230266 with bad export cookie 6741736566685173476 [ 2190.865529] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2190.872324] LustreError: 75469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2191.184593] Lustre: server umount lustre-MDT0001 complete [ 2199.976888] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:11:13 (1781230273) [ 2206.538140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2207.023156] LustreError: 77734:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2207.039247] LustreError: 77734:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2207.143739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2212.022227] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2212.334285] LustreError: 77735:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2217.456479] LustreError: 77734:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2222.034505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2226.474861] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2234.142828] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2239.045261] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2248.044096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2248.711197] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2248.728644] Lustre: Skipped 1 previous similar message [ 2257.295104] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2263.122282] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2267.854386] Lustre: 80069:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 9f28af39-9170-4e34-997d-8f1888cd5a34 at adminstrative request [ 2272.234579] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2272.243640] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2282.476952] Lustre: MGS: Client 9f28af39-9170-4e34-997d-8f1888cd5a34 (at 0@lo) reconnecting [ 2282.493776] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x5d8f75b02a880a56 to 0x5d8f75b02a880cf6 [ 2282.505605] Lustre: MGC192.168.201.102@tcp: Connection restored to 0@lo (at 0@lo) [ 2305.297916] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2305.576498] Lustre: lustre-OST0001: new disk, initializing [ 2305.580320] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2305.586870] Lustre: 80460:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2312.086501] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2315.376550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2315.392749] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2315.530701] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2319.319737] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2326.001207] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2326.005220] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2326.010415] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2326.012618] Lustre: Skipped 2 previous similar messages [ 2326.030214] Lustre: Skipped 5 previous similar messages [ 2336.223148] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2336.534764] Lustre: server umount lustre-OST0001 complete [ 2341.346017] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2341.351551] LustreError: 81203:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2341.363094] Lustre: Skipped 1 previous similar message [ 2341.379472] LustreError: 81203:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2346.467766] LustreError: 79332:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2346.490814] LustreError: 79332:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2346.736094] Lustre: server umount lustre-OST0000 complete [ 2361.826112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2361.840063] Lustre: Skipped 11 previous similar messages [ 2365.407250] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2365.721629] Lustre: server umount lustre-MDT0000 complete [ 2366.955734] LustreError: 77735:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2366.992048] LustreError: 77735:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2370.603476] LustreError: 79352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230445 with bad export cookie 6741736566685175030 [ 2370.608076] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2370.623649] LustreError: 79352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2371.209894] Lustre: server umount lustre-MDT0001 complete [ 2382.153394] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:14:15 (1781230455) [ 2384.584028] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2386.095964] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:14:20 (1781230460) [ 2388.228264] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2390.502935] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:14:23 (1781230463) [ 2392.209284] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2394.191946] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:14:27 (1781230467) [ 2395.795346] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2397.496371] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:14:31 (1781230471) [ 2399.225477] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2406.918790] loop: module loaded [ 2406.950068] loop0: detected capacity change from 0 to 4294967296 [ 2407.038307] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2409.554150] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2413.274559] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2417.430599] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2445.797213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2445.857523] LustreError: 84460:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2445.869721] LustreError: 84460:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2445.881624] LustreError: 84460:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2445.899352] LustreError: 84460:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2466.167891] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 2476.776723] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 2512.511477] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2512.806710] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2512.841235] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2512.917446] Lustre: lustre-MDT0000: new disk, initializing [ 2513.040693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2513.049138] Lustre: Skipped 1 previous similar message [ 2513.076847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2516.986165] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2528.209254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2528.342980] Lustre: 87288:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2528.601422] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2534.093866] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2542.126515] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2546.197286] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2552.728570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2553.013286] Lustre: 89032:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2559.798218] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2562.719772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2562.900834] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2567.820759] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2573.268413] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2573.638325] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2578.794533] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2579.035499] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2583.527137] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2583.529693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2583.538674] LustreError: Skipped 1 previous similar message [ 2583.555740] Lustre: Skipped 2 previous similar messages [ 2583.562667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2587.547479] Lustre: server umount lustre-OST0000 complete [ 2598.334584] Lustre: server umount lustre-MDT0000 complete [ 2598.892423] LustreError: 88634:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2598.908238] LustreError: 88634:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2602.570228] LustreError: 87281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230677 with bad export cookie 6741736566685175646 [ 2602.585196] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2602.596525] LustreError: 87281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2602.825620] Lustre: server umount lustre-MDT0001 complete [ 2611.195168] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:18:04 (1781230684) [ 2618.299114] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2623.221399] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2631.730325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2637.139871] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2642.851875] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2646.646596] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2653.254704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2653.611790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2653.617557] Lustre: Skipped 4 previous similar messages [ 2659.514887] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2665.506397] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2671.226448] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2671.453339] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2676.518625] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2676.752652] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2683.896250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2684.035306] Lustre: lustre-OST0001: new disk, initializing [ 2684.037572] Lustre: Skipped 2 previous similar messages [ 2684.040901] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2684.044342] Lustre: Skipped 3 previous similar messages [ 2684.047942] Lustre: 94295:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2690.092537] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2691.253302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2691.271037] Lustre: Skipped 2 previous similar messages [ 2691.285313] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2691.375272] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2697.988451] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2702.364825] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2702.570761] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2707.080668] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2707.243822] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2726.581860] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2747.974201] Lustre: Failing over lustre-MDT0000 [ 2748.416567] Lustre: server umount lustre-MDT0000 complete [ 2749.934738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2749.939215] LustreError: Skipped 1 previous similar message [ 2749.945443] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2749.968729] Lustre: Skipped 3 previous similar messages [ 2749.974477] LustreError: 92007:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2750.003161] LustreError: 92007:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2769.375724] Lustre: 64380:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781230828/real 1781230828] req@00000000e25bea0a x1867754950104448/t0(0) o400->MGC192.168.201.102@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1781230844 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2769.433538] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2774.536299] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2774.539924] LDISKFS-fs (dm-0): recovery complete [ 2774.545651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2780.157756] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2783.034393] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2783.036723] Lustre: 91291:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2785.069831] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2785.252466] Lustre: 91296:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2785.267125] Lustre: 91296:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 2785.291702] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2785.428831] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2785.760132] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2786.108621] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2786.127913] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2786.472460] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2786.484526] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2786.503518] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2786.540826] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 2787.197798] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2787.243933] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2787.575140] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2787.580507] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2788.553966] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2788.569656] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 5 previous similar messages [ 2789.226409] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2789.235184] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 5 previous similar messages [ 2789.879689] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2789.894102] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2792.741595] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2792.770120] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 2793.432804] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2793.456158] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2794.103152] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2794.116792] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2800.876300] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2800.889227] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 24 previous similar messages [ 2801.545038] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2801.555633] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 2802.199358] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2802.208554] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 2816.992530] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2817.015443] Lustre: 96997:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 49 previous similar messages [ 2817.663965] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2817.681846] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 49 previous similar messages [ 2818.319148] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2818.330259] LustreError: 96997:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 49 previous similar messages [ 2843.070048] Lustre: 96997:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2843.084286] Lustre: 96997:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2843.101549] Lustre: 96997:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 7d253a1a-5c4b-427a-964f-6ad290ceb7b5@192.168.201.2@tcp [ 2843.141835] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2843.164857] Lustre: 96997:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@000000002c0fff21 x1867754933129216/t0(8589934776) o35->7d253a1a-5c4b-427a-964f-6ad290ceb7b5@192.168.201.2@tcp:198/0 lens 392/0 e 0 to 0 dl 1781230928 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2843.209171] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2843.245374] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2843.249146] Lustre: lustre-MDT0000: Denying connection for new client 7d253a1a-5c4b-427a-964f-6ad290ceb7b5 (at 192.168.201.2@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2843.345683] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2843.377257] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2843.379134] Lustre: Skipped 2 previous similar messages [ 2843.501781] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2843.505892] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2865.121085] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2865.135175] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2865.151195] Lustre: Skipped 3 previous similar messages [ 2865.160396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2865.162848] Lustre: Skipped 3 previous similar messages [ 2870.050222] Lustre: server umount lustre-OST0000 complete [ 2872.291663] LustreError: 93397:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2872.325547] LustreError: 93397:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 43 previous similar messages [ 2880.823522] Lustre: server umount lustre-OST0001 complete [ 2891.103967] Lustre: server umount lustre-MDT0000 complete [ 2895.750119] LustreError: 91276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781230970 with bad export cookie 6741736566685226865 [ 2895.759862] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2895.762400] LustreError: 91276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2896.125506] Lustre: server umount lustre-MDT0001 complete [ 2904.803256] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 22:22:58 (1781230978) [ 2913.139540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2913.145148] Lustre: *** cfs_fail_loc=197, val=0*** [ 2928.737690] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2933.856361] LustreError: 98956:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.102@tcp: failed processing log, type 4: rc = -110 [ 2938.989545] Lustre: *** cfs_fail_loc=197, val=0*** [ 2938.991895] Lustre: Skipped 65 previous similar messages [ 2959.583607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2959.593972] Lustre: Skipped 2 previous similar messages [ 2965.511864] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 2970.507960] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2972.339110] Lustre: server umount lustre-OST0000 complete [ 2980.575485] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 22:24:14 (1781231054) [ 3001.502828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3001.936132] LustreError: 100463:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3001.958662] LustreError: 100463:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3006.212917] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3013.962942] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3017.815850] LustreError: 100495:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3017.825921] LustreError: 100495:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3017.830841] LustreError: 100495:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 3018.133827] Lustre: server umount lustre-MDT0000 complete [ 3053.000488] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 3065.863155] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 3106.103100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3106.462075] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3106.502570] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3106.578516] Lustre: lustre-MDT0000: new disk, initializing [ 3106.656689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3111.182755] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3122.461608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3122.617052] Lustre: 104524:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3122.912266] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3127.697118] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3134.541601] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3139.120210] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3146.840267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3147.075439] Lustre: 106265:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3153.598077] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3154.265589] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3162.743775] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3168.327688] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3168.558554] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3173.948618] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3174.354538] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3178.988803] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3178.998239] LustreError: Skipped 2 previous similar messages [ 3179.002431] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3179.018248] Lustre: Skipped 5 previous similar messages [ 3179.023099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3179.027352] Lustre: Skipped 8 previous similar messages [ 3182.341894] Lustre: server umount lustre-OST0000 complete [ 3192.937413] Lustre: server umount lustre-MDT0000 complete [ 3197.327713] LustreError: 104515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231272 with bad export cookie 6741736566685237519 [ 3197.335460] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3197.346803] LustreError: 104515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3197.701772] Lustre: server umount lustre-MDT0001 complete [ 3209.005575] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 22:28:02 (1781231282) [ 3219.283445] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 3222.521341] Key type lgssc unregistered [ 3222.801701] LNet: 108592:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3222.817652] LNetError: 108592:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3223.851541] LNet: Removed LNI 192.168.201.102@tcp [ 3224.879120] Key type .llcrypt unregistered [ 3224.881895] Key type ._llcrypt unregistered [ 3244.016069] Key type ._llcrypt registered [ 3244.017781] Key type .llcrypt registered [ 3244.124346] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 3261.858939] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 3262.958770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3263.015218] alg: No test for adler32 (adler32-zlib) [ 3264.123447] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 3264.320205] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 3266.000114] Key type lgssc registered [ 3267.113451] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3310.285715] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3310.343707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3311.793926] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3311.825805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3311.986494] Lustre: lustre-MDT0000: new disk, initializing [ 3312.127213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3312.149839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3316.128638] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3326.845288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3327.128283] Lustre: lustre-OST0000: new disk, initializing [ 3327.136541] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3327.144411] Lustre: Skipped 1 previous similar message [ 3327.155794] Lustre: 112717:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3327.232793] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3330.021625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3330.037297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3330.069647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3332.663894] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3342.478324] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3342.661876] Lustre: lustre-OST0001: new disk, initializing [ 3342.665487] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3342.673130] Lustre: 113562:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3342.731624] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3348.673750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3348.681460] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3348.706308] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3350.412290] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3359.202959] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3359.206704] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3359.214628] Lustre: Skipped 1 previous similar message [ 3364.326298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3364.332056] Lustre: Skipped 2 previous similar messages [ 3365.838632] Lustre: server umount lustre-MDT0000 complete [ 3372.443676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3376.778440] LustreError: 111920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231451 with bad export cookie 8305943918530255177 [ 3376.779346] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3376.792121] LustreError: 111920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3376.876761] Lustre: server umount lustre-OST0000 complete [ 3380.923678] Lustre: server umount lustre-OST0001 complete [ 3389.486847] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 22:31:02 (1781231462) [ 3399.208431] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 3423.188945] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 22:31:37 (1781231497) [ 3462.768201] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 3471.670888] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 3508.254265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3508.452353] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3508.478115] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3508.570341] Lustre: lustre-MDT0000: new disk, initializing [ 3508.652460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3508.673676] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3512.777474] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3523.704848] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3523.809604] Lustre: 120240:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3523.870781] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3523.874234] Lustre: Skipped 1 previous similar message [ 3523.932696] Lustre: lustre-MDT0001: new disk, initializing [ 3524.038598] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3524.078773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3524.089810] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3528.351291] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3536.188702] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3539.747867] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3546.514497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3546.770578] Lustre: lustre-OST0000: new disk, initializing [ 3546.773880] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3546.779475] Lustre: 121982:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3546.884527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3553.912042] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3553.925549] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3554.024183] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3554.665226] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3563.613260] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3568.924833] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3569.141751] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3573.529411] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3573.750750] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3579.876874] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3579.897834] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3579.910031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3579.924244] Lustre: Skipped 1 previous similar message [ 3582.239269] Lustre: server umount lustre-OST0000 complete [ 3590.115180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3590.129298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3590.133591] Lustre: Skipped 2 previous similar messages [ 3590.142816] Lustre: Skipped 1 previous similar message [ 3593.627900] Lustre: server umount lustre-MDT0000 complete [ 3597.475575] LustreError: 121017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231672 with bad export cookie 8305943918530256402 [ 3597.482506] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3597.488720] LustreError: 121017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3597.897967] Lustre: server umount lustre-MDT0001 complete [ 3605.785613] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 22:34:39 (1781231679) [ 3612.031099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3612.417990] LustreError: 124246:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3612.434490] LustreError: 124246:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3612.474619] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3617.088613] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3617.761710] LustreError: 124247:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3621.867093] LustreError: 124247:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3625.459151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3630.754885] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3636.751617] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3640.487693] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3647.817919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3648.079698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3648.086059] Lustre: Skipped 1 previous similar message [ 3653.467932] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3659.591350] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3664.195955] Lustre: *** cfs_fail_loc=159, val=0*** [ 3680.652474] Lustre: lustre-MDT0000: Client 05924ce8-8b24-44c2-a56b-45f29db15495 (at 192.168.201.2@tcp) reconnecting [ 3682.475658] Lustre: *** cfs_fail_loc=159, val=0*** [ 3682.477128] Lustre: Skipped 5 previous similar messages [ 3698.066407] Lustre: lustre-MDT0000: Client 05924ce8-8b24-44c2-a56b-45f29db15495 (at 192.168.201.2@tcp) reconnecting [ 3703.778686] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3703.789313] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3703.802055] LustreError: Skipped 1 previous similar message [ 3703.805304] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3703.818199] Lustre: Skipped 1 previous similar message [ 3703.843749] Lustre: Skipped 1 previous similar message [ 3709.768864] Lustre: server umount lustre-OST0000 complete [ 3714.018629] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3714.042969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3714.045774] Lustre: Skipped 2 previous similar messages [ 3719.679783] Lustre: server umount lustre-MDT0000 complete [ 3723.542336] LustreError: 124228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231798 with bad export cookie 8305943918530257284 [ 3723.563945] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3723.565614] LustreError: 124228:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3723.854621] Lustre: server umount lustre-MDT0001 complete [ 3735.324624] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 3737.618512] Key type lgssc unregistered [ 3737.886465] LNet: 127665:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3737.890201] LNetError: 127665:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3737.905607] LNet: Removed LNI 192.168.201.102@tcp [ 3738.609820] Key type .llcrypt unregistered [ 3738.612802] Key type ._llcrypt unregistered [ 3764.328585] Key type ._llcrypt registered [ 3764.329789] Key type .llcrypt registered [ 3764.405318] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 3765.847093] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3765.936647] alg: No test for adler32 (adler32-zlib) [ 3767.522994] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 3767.982742] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 3769.799522] Key type lgssc registered [ 3771.727814] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3782.040715] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3782.066351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3784.079304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3789.224469] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3794.407586] LustreError: 129067:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3794.437965] LustreError: 129067:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3799.318813] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3799.524051] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 3799.758849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3804.166793] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3810.398220] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3814.884884] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3821.769132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3822.302870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3828.923885] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 3829.432773] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3834.608702] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3844.210802] Lustre: *** cfs_fail_loc=159, val=0*** [ 3859.812679] Lustre: lustre-MDT0000: Client 43e1f8ec-ac49-4921-8869-52e6df22d099 (at 192.168.201.2@tcp) reconnecting [ 3862.152464] Lustre: *** cfs_fail_loc=159, val=0*** [ 3878.837391] Lustre: lustre-MDT0001: Client 43e1f8ec-ac49-4921-8869-52e6df22d099 (at 192.168.201.2@tcp) reconnecting [ 3881.457088] Lustre: *** cfs_fail_loc=159, val=0*** [ 3881.458793] Lustre: Skipped 3 previous similar messages [ 3897.769027] Lustre: lustre-MDT0001: Client 43e1f8ec-ac49-4921-8869-52e6df22d099 (at 192.168.201.2@tcp) reconnecting [ 3914.095318] Lustre: 131786:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3916.538527] Lustre: *** cfs_fail_loc=159, val=0*** [ 3916.550527] Lustre: Skipped 3 previous similar messages [ 3932.113950] Lustre: lustre-MDT0000: Client e3621068-0805-47b9-9005-b8226ca14d4d (at 192.168.201.2@tcp) reconnecting [ 3935.552809] Lustre: *** cfs_fail_loc=159, val=0*** [ 3935.557104] Lustre: Skipped 14 previous similar messages [ 3950.973544] Lustre: lustre-MDT0000: Client e3621068-0805-47b9-9005-b8226ca14d4d (at 192.168.201.2@tcp) reconnecting [ 3952.127490] Lustre: 132077:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3973.093137] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3973.094144] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3973.114152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3973.122114] Lustre: Skipped 1 previous similar message [ 3973.153532] Lustre: Skipped 1 previous similar message [ 3978.211466] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3978.244147] Lustre: Skipped 1 previous similar message [ 3978.976159] Lustre: server umount lustre-OST0000 complete [ 3984.352822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3984.360046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3984.381559] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3988.452328] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3988.460177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3988.465272] Lustre: Skipped 1 previous similar message [ 3990.196627] Lustre: server umount lustre-MDT0000 complete [ 3993.571088] LustreError: 129066:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3993.590186] LustreError: 129066:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3993.658831] LustreError: 129045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232068 with bad export cookie 2893351580849963143 [ 3993.668137] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3993.675818] LustreError: 129045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3994.002913] Lustre: server umount lustre-MDT0001 complete [ 4008.111552] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 4011.472682] Key type lgssc unregistered [ 4011.977980] LNet: 133141:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4011.983821] LNetError: 133141:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4012.004752] LNet: Removed LNI 192.168.201.102@tcp [ 4012.885615] Key type .llcrypt unregistered [ 4012.887338] Key type ._llcrypt unregistered [ 4040.795662] Key type ._llcrypt registered [ 4040.797922] Key type .llcrypt registered [ 4040.947731] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4042.226839] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4042.308911] alg: No test for adler32 (adler32-zlib) [ 4043.371458] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 4043.561535] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 4045.194351] Key type lgssc registered [ 4046.464576] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4056.256137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4056.270845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4058.213253] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4064.989619] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4068.342218] LustreError: 134539:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4068.360785] LustreError: 134539:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4073.442047] LustreError: 134540:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4075.748523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4076.190739] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4082.721214] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4088.345123] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4093.184125] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4100.888929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4101.098632] LustreError: 136133:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4101.272095] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4103.377625] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4103.621886] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4107.918750] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4115.438122] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4121.856363] Lustre: 136785:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4125.758668] Lustre: 136833:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4127.958834] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4129.437112] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4134.286734] Lustre: 136981:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4135.741799] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4136.913987] Lustre: 137090:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4144.608729] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4144.610522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4144.621989] LustreError: Skipped 1 previous similar message [ 4144.627661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4144.642112] Lustre: Skipped 1 previous similar message [ 4144.661305] Lustre: Skipped 1 previous similar message [ 4148.413982] Lustre: server umount lustre-OST0000 complete [ 4152.799987] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4152.808620] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4152.818748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4154.852805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4154.854234] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4159.970314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4159.976639] Lustre: Skipped 2 previous similar messages [ 4164.066162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4164.077457] Lustre: Skipped 1 previous similar message [ 4167.135528] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4167.426530] Lustre: server umount lustre-MDT0000 complete [ 4169.184572] LustreError: 134536:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4169.218156] LustreError: 134536:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4171.556220] LustreError: 134521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232246 with bad export cookie 11657099203807359974 [ 4171.557714] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4171.567102] LustreError: 134521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4172.135581] Lustre: server umount lustre-MDT0001 complete [ 4182.859943] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 4186.225849] Key type lgssc unregistered [ 4186.528558] LNet: 138126:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4186.534902] LNetError: 138126:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4186.568762] LNet: Removed LNI 192.168.201.102@tcp [ 4187.751071] Key type .llcrypt unregistered [ 4187.753651] Key type ._llcrypt unregistered [ 4213.422125] Key type ._llcrypt registered [ 4213.425472] Key type .llcrypt registered [ 4213.606290] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4215.048230] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4215.387248] alg: No test for adler32 (adler32-zlib) [ 4216.523301] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 4216.821672] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 4218.546037] Key type lgssc registered [ 4219.701299] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4228.051758] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4228.099242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4229.832293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4234.895442] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4239.331274] LustreError: 139525:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4239.356756] LustreError: 139525:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4244.287150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4244.451627] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4244.627273] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4249.403650] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4254.600155] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4258.832096] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4265.614704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4265.909912] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4268.997856] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4271.750821] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4274.096547] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4278.124929] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4284.542535] Lustre: *** cfs_fail_loc=159, val=0*** [ 4284.550528] Lustre: Skipped 1 previous similar message [ 4300.146228] Lustre: lustre-MDT0000: Client c3721f2c-384a-4180-8cc4-12d255e064c5 (at 192.168.201.2@tcp) reconnecting [ 4309.993350] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4310.009972] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4310.020697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4313.252746] Lustre: server umount lustre-OST0000 complete [ 4320.231689] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4320.235286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4320.261148] Lustre: Skipped 2 previous similar messages [ 4320.269669] Lustre: Skipped 2 previous similar messages [ 4322.994588] Lustre: server umount lustre-MDT0000 complete [ 4325.346822] LustreError: 139520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4325.372606] LustreError: 139520:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4327.803246] LustreError: 139505:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232402 with bad export cookie 18200068380662610604 [ 4327.813694] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4327.818580] LustreError: 139505:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4328.207665] Lustre: server umount lustre-MDT0001 complete [ 4339.436951] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 4341.839412] Key type lgssc unregistered [ 4342.133410] LNet: 142828:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4342.146925] LNetError: 142828:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4342.168340] LNet: Removed LNI 192.168.201.102@tcp [ 4342.948761] Key type .llcrypt unregistered [ 4342.950796] Key type ._llcrypt unregistered [ 4366.983176] Key type ._llcrypt registered [ 4366.986619] Key type .llcrypt registered [ 4367.146346] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4368.861222] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4368.898811] alg: No test for adler32 (adler32-zlib) [ 4369.939757] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 4370.134207] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 4371.823254] Key type lgssc registered [ 4372.877428] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4380.325211] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4380.360852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4382.051113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4386.474527] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4392.417468] LustreError: 144227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4392.439237] LustreError: 144227:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4395.588209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4395.907037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4400.423638] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4406.088414] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4410.162282] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4416.653350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4416.962856] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4420.030909] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4423.950296] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4427.190065] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4429.238367] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4449.087844] Lustre: lustre-OST0000: evicting f6982fb2-515a-4a54-b625-b32ec9414a48 (at 192.168.201.2@tcp) by administrative request [ 4449.094674] Lustre: Skipped 1 previous similar message [ 4481.504723] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4481.516108] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4481.525235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4483.559679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4483.560209] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4483.560304] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4483.562549] Lustre: Skipped 1 previous similar message [ 4488.684775] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4488.704865] Lustre: Skipped 1 previous similar message [ 4493.807939] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4493.814907] Lustre: Skipped 1 previous similar message [ 4494.303398] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4494.538611] Lustre: server umount lustre-OST0000 complete [ 4498.914719] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4498.948166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4503.521686] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4503.525169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4504.897805] Lustre: server umount lustre-MDT0000 complete [ 4508.938257] LustreError: 145855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232584 with bad export cookie 7111041741963202745 [ 4508.942689] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4508.951157] LustreError: 145855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4509.287880] Lustre: server umount lustre-MDT0001 complete [ 4520.637449] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 4523.585762] Key type lgssc unregistered [ 4523.910680] LNet: 147946:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4523.917399] LNetError: 147946:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4523.942317] LNet: Removed LNI 192.168.201.102@tcp [ 4524.898425] Key type .llcrypt unregistered [ 4524.908291] Key type ._llcrypt unregistered [ 4600.572976] Key type ._llcrypt registered [ 4600.574818] Key type .llcrypt registered [ 4600.670251] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4602.081229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4602.238349] alg: No test for adler32 (adler32-zlib) [ 4603.415581] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 4603.737222] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 4605.423196] Key type lgssc registered [ 4606.730174] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4615.505898] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4615.530750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4617.276467] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4622.413568] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4627.424904] LustreError: 150416:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4627.443833] LustreError: 150416:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4631.660725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4632.284851] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4636.817394] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4642.424363] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4646.664744] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4655.550660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4656.109548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4657.135598] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4662.264343] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4663.106969] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4669.065227] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4682.726471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4682.730147] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4682.732702] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4682.753724] Lustre: Skipped 1 previous similar message [ 4682.780985] LustreError: Skipped 1 previous similar message [ 4684.520667] Lustre: server umount lustre-OST0000 complete [ 4688.362935] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4688.370907] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4688.390914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4688.393552] Lustre: Skipped 1 previous similar message [ 4692.961356] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4692.964060] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4694.522817] LustreError: 152868:0:(ldlm_resource.c:1180:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (00000000e0ad9f74) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 4694.667461] Lustre: server umount lustre-MDT0000 complete [ 4698.049250] LustreError: 150397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232773 with bad export cookie 18318325956180023257 [ 4698.052362] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4698.070264] LustreError: 150397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4698.477553] Lustre: server umount lustre-MDT0001 complete [ 4708.419536] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 4710.835519] Key type lgssc unregistered [ 4711.039255] LNet: 153602:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4711.047860] LNetError: 153602:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4711.057743] LNet: Removed LNI 192.168.201.102@tcp [ 4711.885888] Key type .llcrypt unregistered [ 4711.891774] Key type ._llcrypt unregistered [ 4736.366797] Key type ._llcrypt registered [ 4736.369681] Key type .llcrypt registered [ 4736.494512] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4737.940804] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4738.054368] alg: No test for adler32 (adler32-zlib) [ 4739.300180] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 4739.720788] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 4741.520717] Key type lgssc registered [ 4743.006738] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4758.735902] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:53:52 (1781232832) [ 4775.120483] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 4784.095205] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4822.792646] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4822.828234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4824.201497] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4824.255476] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4824.381554] Lustre: lustre-MDT0000: new disk, initializing [ 4824.525750] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4824.551481] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4829.149366] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4840.687993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4840.835283] Lustre: 157814:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4840.928846] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4840.940036] Lustre: Skipped 1 previous similar message [ 4841.051547] Lustre: lustre-MDT0001: new disk, initializing [ 4841.216264] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4841.272586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4841.287879] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4847.029490] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4856.447382] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4861.494452] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4868.760542] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4869.091727] Lustre: lustre-OST0000: new disk, initializing [ 4869.093791] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4869.098832] Lustre: 159554:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4869.193936] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4875.954327] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4876.336125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4876.349287] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4876.475371] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4883.807528] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4896.738154] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4896.745107] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4896.782172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4897.262155] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4897.276682] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4897.303528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4897.321812] Lustre: Skipped 1 previous similar message [ 4901.234754] Lustre: server umount lustre-OST0000 complete [ 4907.491045] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4907.504795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4912.610539] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4912.615391] Lustre: Skipped 2 previous similar messages [ 4917.729296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4919.265302] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4919.614793] Lustre: server umount lustre-MDT0000 complete [ 4924.604906] LustreError: 160394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232999 with bad export cookie 16311132084892408996 [ 4924.617726] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4924.965714] Lustre: server umount lustre-MDT0001 complete [ 4936.596416] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 4939.886495] Key type lgssc unregistered [ 4940.223893] LNet: 161334:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4940.232986] LNetError: 161334:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4940.250657] LNet: Removed LNI 192.168.201.102@tcp [ 4941.258177] Key type .llcrypt unregistered [ 4941.262541] Key type ._llcrypt unregistered [ 4967.592907] Key type ._llcrypt registered [ 4967.595390] Key type .llcrypt registered [ 4967.690592] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 4968.785041] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4968.951333] alg: No test for adler32 (adler32-zlib) [ 4970.205746] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 4970.520373] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 4972.280321] Key type lgssc registered [ 4973.722384] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4981.310661] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4981.332814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4982.727840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4987.129681] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 4991.975374] LustreError: 162735:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4992.003386] LustreError: 162735:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4996.386029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4996.780426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5000.648573] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 5006.305067] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5010.420528] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5018.022322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5018.375482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5024.402460] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 5027.571454] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5029.830705] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5036.125688] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5036.150597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5036.165833] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5036.178867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.200565] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5037.303985] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5037.347499] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5037.367817] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5037.379515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.391922] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5038.278099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.123309] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5039.127474] Lustre: Skipped 1 previous similar message [ 5039.140088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5039.148416] Lustre: Skipped 1 previous similar message [ 5039.152119] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5039.155301] Lustre: Skipped 1 previous similar message [ 5039.159438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.167521] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5039.177526] Lustre: Skipped 1 previous similar message [ 5040.109355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.928287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.102394] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5042.105459] Lustre: Skipped 2 previous similar messages [ 5042.127486] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5042.145405] Lustre: Skipped 2 previous similar messages [ 5042.158367] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5042.164557] Lustre: Skipped 2 previous similar messages [ 5042.169346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.180153] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5042.192327] Lustre: Skipped 2 previous similar messages [ 5043.087764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.068293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.789536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.730576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.680310] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5046.688561] Lustre: Skipped 4 previous similar messages [ 5046.713941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5046.724087] Lustre: Skipped 4 previous similar messages [ 5046.730620] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5046.740377] Lustre: Skipped 4 previous similar messages [ 5046.742737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.770071] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5046.776382] Lustre: Skipped 4 previous similar messages [ 5047.764516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.624420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.501339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.411669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.426561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.171261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.041276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.760154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.756320] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5054.763707] Lustre: Skipped 8 previous similar messages [ 5054.780705] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5054.791350] Lustre: Skipped 8 previous similar messages [ 5054.799731] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5054.805606] Lustre: Skipped 8 previous similar messages [ 5054.808946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.821085] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5054.825899] Lustre: Skipped 8 previous similar messages [ 5055.655932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.582674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.397821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.382818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.288380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.185067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.175506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.974209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.915319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.153351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.152418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.166683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.109615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.995327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.747298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.657398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.391106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.333951] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5071.339981] Lustre: Skipped 17 previous similar messages [ 5071.355491] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5071.363661] Lustre: Skipped 17 previous similar messages [ 5071.367531] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5071.371471] Lustre: Skipped 17 previous similar messages [ 5071.374734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.386285] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5071.412404] Lustre: Skipped 17 previous similar messages [ 5072.321426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.233166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.049896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.115680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.969382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.893816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.883504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.894823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.863343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.845145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.833904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.727420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.639155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.704135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.612919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.476851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.400353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.345653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.107369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.990168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.862855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.105269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.953787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.991912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.795846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.671609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.505422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.353522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.317269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.176291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.101244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.174939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.921921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.751894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.738294] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5103.747027] Lustre: Skipped 34 previous similar messages [ 5103.815399] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5103.831940] Lustre: Skipped 34 previous similar messages [ 5103.836372] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5103.848849] Lustre: Skipped 34 previous similar messages [ 5103.852933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.859457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5103.863309] Lustre: Skipped 34 previous similar messages [ 5104.630122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.440835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.405860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.298345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.528355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.498517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.538800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.653578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.438348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.482240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.478876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.279822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.051495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.066878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.947728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.985701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.877845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.848490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.617418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.515600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.692610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.796503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.801403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.637693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.414812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.516204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.577061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.965624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.581237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.562562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.602197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.877681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.984751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.149237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.961727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.214393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.119702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.166166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.900391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.804615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.526859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.392167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.387311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.699098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.314039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.415831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.575357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.516850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.836673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.692574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.873215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.078065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.122105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.888303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.773472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.758708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.790658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.601278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.262636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.042346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.732532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.662831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.641953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.675151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.667230] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5168.671876] Lustre: Skipped 64 previous similar messages [ 5168.683761] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5168.696283] Lustre: Skipped 64 previous similar messages [ 5168.701319] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5168.703730] Lustre: Skipped 64 previous similar messages [ 5168.708471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.717754] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5168.724695] Lustre: Skipped 64 previous similar messages [ 5169.522476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.631680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.770666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.897339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.167322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.499747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.464472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.597745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.615446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.538564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.317439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.464712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.129921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.026228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.797823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.896847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.809112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.844038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.975931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.804098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.858600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.808122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.825930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.749215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.682762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.715676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.474071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.430886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.171335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.204145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.973919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.953784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.678941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.655735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.351775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.178175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.997546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.872063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.615176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.656773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.424920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.320894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.083748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.050371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.865026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.766630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.551899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.351561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.310323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.033566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.811644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.804453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.655218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.561960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.283568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.318486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.122043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.032960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.891983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.611104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.272968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.260485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.017164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.840191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.683759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.489082] LustreError: 162755:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5228.495987] LustreError: 162755:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5228.503220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.140412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.127939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.951242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.766358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.505746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.209624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.997986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.756955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.621841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.577537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.405366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.130370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.799079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.610937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.559793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.495915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.507436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.380049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.236835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.443785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.463872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.401832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.383394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.541956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.603883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.879304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.027964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.960610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.142972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.104936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.145769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.026873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.302353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.333608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.060921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.796700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.889614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.865880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.556463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.518777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.645255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.804788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.973990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.338355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.285928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.379285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.325675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.304567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.408971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.280327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.542622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.595790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.622257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.763579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.864119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.799848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.736362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.785549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.519370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.712436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.631272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.784989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.548540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.230320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.188782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.177777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.852211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.694417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.549179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.242819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.946706] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5296.950453] Lustre: Skipped 136 previous similar messages [ 5296.957942] LustreError: 162755:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5296.970978] LustreError: 162755:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5296.976368] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5296.996371] Lustre: Skipped 136 previous similar messages [ 5296.998943] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5297.001804] Lustre: Skipped 136 previous similar messages [ 5297.005788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.013842] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5297.021713] Lustre: Skipped 136 previous similar messages [ 5297.814950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.737191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.602245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.523460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.414246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.338206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.184032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.027548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.905734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.948505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.852176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.863235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.616864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.464760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.435809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.596933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.580656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.656294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.660688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.537713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.470107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.422330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.446862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.663880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.640669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.740351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.695313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.439804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.434062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.931416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.944424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.262533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.567628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.909053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.226570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.491450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.710678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.503979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.532426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.537300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.560756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.477079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.333152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.294834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.173832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.279188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.218787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.044779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.987448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.796496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.918602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.875525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.025647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.895182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.112309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.289213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.341317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.349081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.454094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.487862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.678801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.655910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.725489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.705464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.654942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.483128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.572892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.523891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.592202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.488611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.781926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.990612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.003134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.028690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.858390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.962530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.988500] LustreError: 162755:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5375.006587] LustreError: 162755:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5375.020582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.998815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.354787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.376640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.541155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.706465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.800647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.979289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.102967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.922982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.145314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.201469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.523852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.621710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.632654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.583315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.539564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.607455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.548576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.438187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.122265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.047301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.960905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.919159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.842176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.933784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.008539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.901813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.115318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.032061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.027302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.923573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.867368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.830806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.738278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.800377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.608558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.674653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.453668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.501620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.624096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.838758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.827647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.734693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.618914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.781453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.887666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.842279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.746290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.584264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.430242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.116464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.093945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.057603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.912066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.810874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.084538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.933314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.985727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.807831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.874931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.634389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.505347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.304243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.280691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.224234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.379289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.094559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.036612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.077724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.879933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.065902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.397970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.570877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.748546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.865769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.070059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.038500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.387630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.254243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.440343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.405247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.408272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.203032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.232057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.361278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.623966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.840847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.144323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.377677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.424128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.529643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.702937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.712604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.641651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.693848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.620724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.709171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.636630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.609346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.557378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.577544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.601248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.660244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.780838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.669376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.677169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.695758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.579898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.676843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.689376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.586122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.577338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.717909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.633461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.660332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.522153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.258087] LustreError: 162755:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5493.271393] LustreError: 162755:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5493.279146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.207223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.085244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.876370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.640977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.380798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.398899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.357722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.140465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.885134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.744189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.618664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.698162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.998875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.000084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.053490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.053538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.157894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.161072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.205444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.164133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.994572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.877461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.779823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.669072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.527648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.418857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.214895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.115209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.827363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.699441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.688753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.465075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.161929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.063668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.012648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.776042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.524622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.485528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.415769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.232735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.020368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.807255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.644399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.564753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.582685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.551582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.418949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.238903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.015335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.776674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.908864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.737588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.647594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.502245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.452252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.269384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.077796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.810821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.810284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.617580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.557173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.378282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.255836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.184966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.016779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.851529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.768585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.526747] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5553.532053] Lustre: Skipped 261 previous similar messages [ 5553.553960] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.564627] Lustre: Skipped 261 previous similar messages [ 5553.576426] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5553.580713] Lustre: Skipped 261 previous similar messages [ 5553.586346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.595604] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5553.601361] Lustre: Skipped 261 previous similar messages [ 5554.407765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.201747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.009080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.995629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.178644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.989600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.882736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.760753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.700340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.630139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.567845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.357320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.390717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.218304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.052649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.718299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.447703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.174854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.050070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.962703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.769164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.575388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.494048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.288899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.044770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.893030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.835746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.796709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.781561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.787861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.656727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.568257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.457653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.199757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.048103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.131688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.018366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.955477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.815984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.781744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.713667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.676466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.572601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.442082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.256284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.201080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.178852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.147358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.135055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.121849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.999437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.834697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.661122] LustreError: 162755:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5600.665913] LustreError: 162755:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5600.669045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.411606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.362387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.347907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.363493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.174303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.902463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.818938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.691741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.672734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.671941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.772369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.752366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.742995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.498402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.425178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.397951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.324490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.218649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.249039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.130563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.138512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.997569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.159082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.312752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.515046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.850109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.813308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.142835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.185056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.069360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.830806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.726703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.630736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.505894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.487262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.371636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.212590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.988344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.784759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.681795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.884885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.790972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.857757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.908356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.203082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.353423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.302213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.352408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.137778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.976992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.667670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.516229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.641402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.651711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.421799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.186683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.052922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.893893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.754203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.730977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.553676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.609083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.420048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.346862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.203639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.124940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.066110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.901558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.690187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.711092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.659930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.595991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.305801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.075828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.829908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.571183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.356539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.155842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.031078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.734263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.685514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.588439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.494909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.249958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.138934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.968963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.838101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.680548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.479869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.343265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.361437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.224722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.094503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.092943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.067373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.501858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.759141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.961485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.367692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.969162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.222366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.802516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.330988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.783944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.387307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.465130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.528576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.407417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.258733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.353856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.447225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.854395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.425815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.632189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.562610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.655614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.454756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.652117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.838156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.016382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.066651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.036496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.885215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.956466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.767226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.701854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.771908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.616872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.429378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.443782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.526064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.509647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.566502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.493244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.457723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.275929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.164249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.042169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.114387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.007353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.020077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.125362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.029396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.909656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.802446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.789933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.496977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.528576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.434839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.329687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.232295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.033280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.713216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.579707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.558377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.382208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.099783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.764901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.446569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.288194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.104448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.855076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.567082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.571684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.555182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.776293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.986620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.113241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.905290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.927995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.799957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.767084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.654440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.717458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.669611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.513586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.334931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.292294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.090179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.996275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.868903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.646413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.685719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.962270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.238880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.271566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.446664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.567404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.612555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.668978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.634321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.731523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.717172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.617538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.553723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.548915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.499547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.429532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.358227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.276545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.069759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.946443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.817583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.809478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.863184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.694637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.471751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.330716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.209617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.163340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.177397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.839422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.738910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.650274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.642890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.397533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.297075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.118801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.062633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.867599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.722564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.430747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.241763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.937057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.888813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.748181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.676665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.443871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.259260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.190713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.092536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.863887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.684797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.474652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.286608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.149333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.020815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.857776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.822035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.653214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.465377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.123638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.847275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.596693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.465931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.277830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.147038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.313382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.342770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.132791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.872502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.755962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.535198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.663809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.695322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.715603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.486811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.476899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.443952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.730365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.064406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.143859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.593129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.906113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.090337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.052683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.190653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.045897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.109626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.481315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.946132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.099166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.404362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.590940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.503692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.578885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.848549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.762066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.821213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.839829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.941503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.888936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.067118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.953402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.057498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.200877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.216331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.379320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.599792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.647887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.437619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.409405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.428350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.345393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.287867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.236068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.169585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.110520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.027470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.063373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.071213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.362265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.483327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.459567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.687919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.950732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.665989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.670993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.528598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.679234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.581101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.652947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.620532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.845777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.843266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.032773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.336992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.672365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.862657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.899287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.997942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.023200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.471718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.589407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.461173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.504655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.409783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.408247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.502208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.816262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.148620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.402570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.688775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.854545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.071354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.128392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.114796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.096714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.143855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.097713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.083430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.908860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.974524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.854826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.932263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.775403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.632963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.532440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.449049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.324800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.360939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.433427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.410266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.429318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.486321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.776993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.893332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.802660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.005396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.200634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.269374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.326564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.500759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.466651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.461362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.644377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.049478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.278573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.707190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.118953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.106572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.305933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.214501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.142274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.198471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.309719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.198913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.263573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.188627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.412768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.198568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.344932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.463458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.485552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.360079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.520383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.530483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.379797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.385970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.546332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.454037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.415924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.194229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.216175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.310647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.352736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.397603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.408934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.398174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.524298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.601981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.801644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.960039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.890092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.189289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.239742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.509837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.748372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.116299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.645611] LustreError: 164331:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6324.658251] LustreError: 162755:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6744.544465] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6744.549039] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6744.581570] Lustre: Skipped 462 previous similar messages [ 6744.605900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6748.687573] Lustre: server umount lustre-OST0000 complete [ 6752.738285] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6752.741501] LustreError: Skipped 2 previous similar messages [ 6752.758000] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6752.768307] Lustre: Skipped 2 previous similar messages [ 6755.298198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6755.307993] Lustre: Skipped 1 previous similar message [ 6759.207795] Lustre: server umount lustre-MDT0000 complete [ 6760.430382] LustreError: 180854:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6760.458469] LustreError: 180854:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6764.081518] LustreError: 162716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234839 with bad export cookie 5809436010637505344 [ 6764.098695] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6764.100860] LustreError: 162716:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6764.841900] Lustre: server umount lustre-MDT0001 complete [ 6776.907617] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 6779.425163] Key type lgssc unregistered [ 6779.703579] LNet: 217594:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6779.712460] LNetError: 217594:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6780.773825] LNet: Removed LNI 192.168.201.102@tcp [ 6781.701644] Key type .llcrypt unregistered [ 6781.704603] Key type ._llcrypt unregistered [ 6807.856631] Key type ._llcrypt registered [ 6807.858333] Key type .llcrypt registered [ 6807.996763] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 6809.249610] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6809.298513] alg: No test for adler32 (adler32-zlib) [ 6810.464374] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 6810.654321] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 6812.367166] Key type lgssc registered [ 6813.816337] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6824.207942] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6824.238560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6826.496799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6832.758442] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 6836.715937] LustreError: 218995:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6836.762581] LustreError: 218995:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6841.837720] LustreError: 218996:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6844.914662] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6845.495488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6851.197725] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 6857.277190] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6861.570817] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6868.492516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6868.836043] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6870.913703] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6876.263741] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 6882.088825] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6887.959779] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6889.802693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6889.829404] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6889.847698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.863846] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6889.866090] LustreError: 219015:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6894.936579] Lustre: lustre-OST0000: Not available for connect from 192.168.201.2@tcp (stopping) [ 6895.589325] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6895.605793] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6895.626441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6896.611462] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6896.638665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6896.645897] Lustre: Skipped 1 previous similar message [ 6900.033268] Lustre: lustre-OST0000: Not available for connect from 192.168.201.2@tcp (stopping) [ 6905.179269] Lustre: lustre-OST0000: Not available for connect from 192.168.201.2@tcp (stopping) [ 6905.192846] Lustre: Skipped 2 previous similar messages [ 6906.340260] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6906.690279] Lustre: server umount lustre-OST0000 complete [ 6909.930904] LustreError: 219015:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6941.489568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6941.853026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6949.007624] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 6952.120178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.139201] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6955.780285] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6957.286797] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.306242] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6962.872872] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 5 sec [ 6966.494799] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff891f42911000.ost_server_uuid 50 [ 6968.200255] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff891f42911000.ost_server_uuid in FULL state after 0 sec [ 6972.901335] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6972.907791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6972.919598] Lustre: Skipped 1 previous similar message [ 6972.934642] Lustre: Skipped 1 previous similar message [ 6974.390236] Lustre: server umount lustre-OST0000 complete [ 6978.033931] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6978.046102] Lustre: Skipped 1 previous similar message [ 6992.351508] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6992.591449] Lustre: server umount lustre-MDT0000 complete [ 6993.384286] LustreError: 219902:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6993.408529] LustreError: 219902:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6996.530690] LustreError: 218976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235071 with bad export cookie 5667285000980571341 [ 6996.550928] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6996.562438] LustreError: 218976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6997.080590] Lustre: server umount lustre-MDT0001 complete [ 7006.900660] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 7009.589347] Key type lgssc unregistered [ 7009.843949] LNet: 223639:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7009.850658] LNetError: 223639:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7009.867608] LNet: Removed LNI 192.168.201.102@tcp [ 7010.815925] Key type .llcrypt unregistered [ 7010.818162] Key type ._llcrypt unregistered [ 7044.561493] Key type ._llcrypt registered [ 7044.564826] Key type .llcrypt registered [ 7044.681608] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 7045.977330] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7046.028817] alg: No test for adler32 (adler32-zlib) [ 7047.166158] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 7047.446061] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 7049.127288] Key type lgssc registered [ 7050.198916] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7058.852785] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7058.892371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7060.596493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7065.518472] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7070.700081] LustreError: 225352:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7070.726195] LustreError: 225352:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7074.244666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7074.706532] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7079.003993] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7081.836453] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7088.264318] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7088.689668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7096.101632] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7105.161345] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7105.758855] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7106.039435] Lustre: lustre-OST0001: new disk, initializing [ 7106.042976] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7106.052834] Lustre: 227609:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7106.163302] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7112.458675] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7112.818919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7112.831768] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7112.935584] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7124.535210] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7132.057229] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7149.029385] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7149.041737] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7149.053723] Lustre: Skipped 2 previous similar messages [ 7149.059680] Lustre: Skipped 1 previous similar message [ 7153.803197] Lustre: server umount lustre-MDT0000 complete [ 7154.153171] LustreError: 226822:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7154.170757] LustreError: 226822:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7157.292033] LustreError: 225334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235232 with bad export cookie 4161891861983457284 [ 7157.295297] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7157.303603] LustreError: 225334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7157.745909] Lustre: server umount lustre-MDT0001 complete [ 7171.286618] Lustre: server umount lustre-OST0000 complete [ 7186.021686] Lustre: server umount lustre-OST0001 complete [ 7204.918827] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 7215.072518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7215.445038] LustreError: 230683:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7215.546308] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7219.620597] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7220.712323] LustreError: 230684:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7225.839514] LustreError: 230683:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7227.361247] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7227.645641] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7231.669762] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7235.075199] Lustre: 231791:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7242.538382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7250.233264] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7252.204563] LustreError: 232143:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7252.215160] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7259.155544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7259.508751] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7259.519704] Lustre: Skipped 1 previous similar message [ 7261.566430] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7261.566849] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7266.357978] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7274.163718] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7277.251608] Lustre: 233626:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7289.313096] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7289.317258] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7289.359208] Lustre: Skipped 1 previous similar message [ 7289.366076] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7289.374178] Lustre: Skipped 2 previous similar messages [ 7292.387536] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7292.391414] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7292.412462] Lustre: Skipped 1 previous similar message [ 7292.426642] Lustre: Skipped 1 previous similar message [ 7294.623466] Lustre: server umount mylustre-MDT0000 complete [ 7297.509348] LustreError: 231307:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7297.536403] LustreError: 231307:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7298.644258] LustreError: 233825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235373 with bad export cookie 4161891861983460140 [ 7298.646588] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7298.658895] LustreError: 233825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7299.140083] Lustre: server umount mylustre-MDT0001 complete [ 7313.614122] Lustre: server umount mylustre-OST0000 complete [ 7327.666043] Lustre: server umount mylustre-OST0001 complete [ 7352.597820] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 7363.482046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7363.980413] LustreError: 235920:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7364.077414] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7368.221227] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7377.305947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7381.806092] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7384.876848] Lustre: 237029:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7391.470152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7398.558564] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7402.055471] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7408.662526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7409.994141] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7410.000198] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7415.809842] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7423.281974] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7426.798660] Lustre: 238867:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7434.208892] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7434.218901] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7434.231955] Lustre: Skipped 1 previous similar message [ 7434.239194] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7434.250581] Lustre: Skipped 2 previous similar messages [ 7439.481297] Lustre: server umount tfs-MDT0000 complete [ 7440.871637] LustreError: 235921:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7440.900931] LustreError: 235921:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 7443.437299] LustreError: 235897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235518 with bad export cookie 4161891861983462324 [ 7443.445567] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7443.452236] LustreError: 235897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7443.980363] Lustre: server umount tfs-MDT0001 complete [ 7458.070130] Lustre: server umount tfs-OST0000 complete [ 7462.176114] Lustre: 224616:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781235521/real 1781235521] req@ffff8f9ac4f8cf00 x1867760399500160/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781235537 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7462.200456] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7462.212373] Lustre: Skipped 3 previous similar messages [ 7463.874591] Lustre: server umount tfs-OST0001 complete [ 7486.091447] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 7498.800492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7499.508458] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7499.515226] Lustre: Skipped 3 previous similar messages [ 7505.533531] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7517.030397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7524.776371] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7528.036797] Lustre: 242211:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7534.662183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7541.826349] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7543.234949] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7544.246876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7551.618780] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7557.121069] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 7558.638644] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7567.309803] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7571.189585] Lustre: 244048:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7579.158685] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 23:40:52 (1781235652) [ 7598.049464] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7598.062565] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7598.072960] Lustre: Skipped 1 previous similar message [ 7598.096050] Lustre: Skipped 7 previous similar messages [ 7608.297190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7608.308140] Lustre: Skipped 7 previous similar messages [ 7611.363569] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7611.597401] Lustre: server umount lustre-MDT0000 complete [ 7613.412186] LustreError: 242001:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7613.437679] LustreError: 242001:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 7615.787336] LustreError: 241078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235690 with bad export cookie 4161891861983464242 [ 7615.794932] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7615.812073] LustreError: 241078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7618.528901] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7618.544788] Lustre: Skipped 3 previous similar messages [ 7625.702882] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7625.719405] Lustre: Skipped 5 previous similar messages [ 7630.303320] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7630.625556] Lustre: server umount lustre-MDT0001 complete [ 7640.747892] Lustre: server umount lustre-OST0000 complete [ 7650.481842] Lustre: server umount lustre-OST0001 complete [ 7657.339920] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 7668.358804] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 7713.087442] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7713.087442] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7713.087442] [ 7713.127708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7713.332413] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7713.392479] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7713.479570] Lustre: lustre-MDT0000: new disk, initializing [ 7713.580484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7713.590479] Lustre: Skipped 3 previous similar messages [ 7713.628919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7719.257553] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7731.613989] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7731.613989] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7731.613989] [ 7731.651413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7731.756623] Lustre: 247202:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7731.829524] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7731.833440] Lustre: Skipped 1 previous similar message [ 7731.898788] Lustre: lustre-MDT0001: new disk, initializing [ 7732.078540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7732.090407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7737.477725] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7750.217329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7750.743779] Lustre: lustre-OST0000: new disk, initializing [ 7750.757911] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7750.778828] Lustre: 248754:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7752.445714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7752.470279] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7752.617933] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7759.199945] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7771.022854] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7771.182226] Lustre: lustre-OST0001: new disk, initializing [ 7771.185736] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7771.190621] Lustre: 249608:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7773.212359] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7773.218430] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7773.293835] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7777.636342] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7790.381188] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 23:44:23 (1781235863) [ 7811.767612] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 23:44:45 (1781235885) [ 7829.481248] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7829.483953] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7829.491539] Lustre: Skipped 1 previous similar message [ 7829.519447] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7829.522955] Lustre: Skipped 3 previous similar messages [ 7841.247158] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7841.470768] Lustre: server umount lustre-OST0000 complete [ 7849.965878] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7849.992427] LustreError: Skipped 1 previous similar message [ 7853.379650] Lustre: server umount lustre-MDT0000 complete [ 7856.628836] LustreError: 247196:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235931 with bad export cookie 4161891861983466020 [ 7856.630742] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7856.636588] LustreError: 247196:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7863.161292] Lustre: server umount lustre-MDT0001 complete [ 7872.610907] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 7881.948316] Lustre: server umount lustre-OST0001 complete [ 7888.097892] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 7903.512778] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 7941.376737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7941.666987] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7941.692871] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7941.767269] Lustre: lustre-MDT0000: new disk, initializing [ 7941.894097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7946.557153] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7958.093864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7958.237883] Lustre: 254118:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7958.294865] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7958.297432] Lustre: Skipped 1 previous similar message [ 7958.402177] Lustre: lustre-MDT0001: new disk, initializing [ 7958.603183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7958.614385] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7963.724619] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7973.079760] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7977.533819] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7983.885561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7984.152679] Lustre: 255860:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7984.234168] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7984.245381] Lustre: Skipped 5 previous similar messages [ 7985.460569] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7985.660697] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7990.481549] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 7998.403517] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8006.112595] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8006.134559] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8006.138877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8006.147174] Lustre: Skipped 5 previous similar messages [ 8006.156229] Lustre: Skipped 10 previous similar messages [ 8010.169664] Lustre: server umount lustre-OST0000 complete [ 8014.816503] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8014.822835] LustreError: Skipped 1 previous similar message [ 8028.127230] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8028.369285] Lustre: server umount lustre-MDT0000 complete [ 8031.719284] LustreError: 256378:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8031.743742] LustreError: 256378:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 18 previous similar messages [ 8032.320189] LustreError: 254111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236107 with bad export cookie 4161891861983467490 [ 8032.325918] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8032.344426] LustreError: 254111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8033.014759] Lustre: server umount lustre-MDT0001 complete [ 8044.381396] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 8047.155383] Key type lgssc unregistered [ 8047.447657] LNet: 257491:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8047.457811] LNetError: 257491:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8047.479917] LNet: Removed LNI 192.168.201.102@tcp [ 8048.438037] Key type .llcrypt unregistered [ 8048.439667] Key type ._llcrypt unregistered [ 8075.066292] Key type ._llcrypt registered [ 8075.072977] Key type .llcrypt registered [ 8075.220442] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 8076.836742] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8076.862621] alg: No test for adler32 (adler32-zlib) [ 8077.997925] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 8078.210369] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 8079.839283] Key type lgssc registered [ 8081.644471] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8089.824506] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8089.843598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8091.296264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8096.036613] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8100.848298] LustreError: 258955:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8100.875764] LustreError: 258955:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8105.955971] LustreError: 258956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8107.681969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8108.182368] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8112.850129] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8119.387948] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8123.242359] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8130.160615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8130.583152] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8136.755607] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8137.122737] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8142.983796] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8144.308183] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8146.915922] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8146.944257] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8146.944619] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8146.976309] Lustre: Skipped 1 previous similar message [ 8152.038709] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8152.043474] Lustre: Skipped 1 previous similar message [ 8152.867542] Lustre: server umount lustre-OST0000 complete [ 8157.153574] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8157.160554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8157.171890] Lustre: Skipped 2 previous similar messages [ 8157.187406] Lustre: Skipped 1 previous similar message [ 8162.273687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8163.742319] Lustre: server umount lustre-MDT0000 complete [ 8167.402961] LustreError: 258956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8167.431427] LustreError: 258956:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8168.594748] LustreError: 258936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236243 with bad export cookie 2805851093386706818 [ 8168.600060] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8168.603744] LustreError: 258936:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8168.899981] Lustre: server umount lustre-MDT0001 complete [ 8180.961852] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 8183.717720] Key type lgssc unregistered [ 8183.964611] LNet: 262186:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8183.972235] LNetError: 262186:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8183.984217] LNet: Removed LNI 192.168.201.102@tcp [ 8185.004901] Key type .llcrypt unregistered [ 8185.012676] Key type ._llcrypt unregistered [ 8202.656755] Key type ._llcrypt registered [ 8202.659627] Key type .llcrypt registered [ 8202.746468] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 8204.065758] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8204.103961] alg: No test for adler32 (adler32-zlib) [ 8205.222545] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 8205.515066] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 8207.199888] Key type lgssc registered [ 8208.095152] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8216.540904] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8216.554297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8232.160378] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8237.283055] LustreError: 263363:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.102@tcp: failed processing log, type 4: rc = -110 [ 8264.095156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8270.465294] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8275.754139] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8278.038958] Lustre: server umount lustre-OST0000 complete [ 8294.574278] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 8296.841235] Key type lgssc unregistered [ 8297.084642] LNet: 264862:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8297.092397] LNetError: 264862:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8297.114992] LNet: Removed LNI 192.168.201.102@tcp [ 8297.988542] Key type .llcrypt unregistered [ 8297.989729] Key type ._llcrypt unregistered [ 8334.507244] Key type ._llcrypt registered [ 8334.509121] Key type .llcrypt registered [ 8334.593543] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 8335.920385] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8336.088308] alg: No test for adler32 (adler32-zlib) [ 8337.251433] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 8337.419465] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 8339.039197] Key type lgssc registered [ 8340.561782] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8353.819679] loop0: detected capacity change from 0 to 204800000 [ 8354.134661] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8354.173437] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8354.634690] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8355.967835] loop0: detected capacity change from 0 to 204800000 [ 8356.374427] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8356.379758] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8356.678960] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8357.777786] loop0: detected capacity change from 0 to 204800000 [ 8357.976290] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8357.986119] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8358.149859] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8359.600824] loop0: detected capacity change from 0 to 204800000 [ 8359.886201] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8359.930923] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8360.140763] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8361.391056] loop0: detected capacity change from 0 to 204800000 [ 8361.426108] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8367.469669] loop0: detected capacity change from 0 to 204800000 [ 8367.522146] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8371.468428] loop0: detected capacity change from 0 to 204800000 [ 8371.477702] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8375.243834] loop0: detected capacity change from 0 to 204800000 [ 8375.286238] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8379.197476] loop0: detected capacity change from 0 to 204800000 [ 8379.266437] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8379.325189] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8379.398725] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8379.403428] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8379.414192] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8379.426197] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8379.433384] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8379.441751] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8379.452769] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8379.464138] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8379.472608] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8379.479386] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8379.486514] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8379.494507] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8379.502613] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8379.510519] Lustre: 268263:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8385.395898] Lustre: server umount MGS complete [ 8386.394547] loop0: detected capacity change from 0 to 204800000 [ 8386.545120] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8387.292654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8387.355693] Lustre: lustre-MDT0000: Aborting client recovery [ 8388.185763] loop1: detected capacity change from 0 to 204800000 [ 8388.445422] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8388.508981] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8388.551128] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8388.567287] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8388.589292] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8388.603489] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8388.628415] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8388.641299] Lustre: 268694:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8388.700199] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8388.993105] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8389.063560] Lustre: lustre-MDT0001: Aborting client recovery [ 8389.962803] loop2: detected capacity change from 0 to 204800000 [ 8390.157189] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8390.241640] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8390.251786] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8390.267291] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8390.273564] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8390.284658] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8390.289442] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8390.294491] Lustre: 268773:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8390.377271] Lustre: Found index 0 for lustre-OST0000, updating log [ 8390.398923] Lustre: 268791:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8390.454909] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8390.464536] Lustre: lustre-OST0000: Aborting recovery [ 8391.417392] loop3: detected capacity change from 0 to 204800000 [ 8391.638450] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8391.811341] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8391.833046] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8391.847221] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8391.868277] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8391.884207] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8391.900834] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8391.922591] Lustre: 268858:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8391.968075] Lustre: Found index 1 for lustre-OST0001, updating log [ 8392.040476] Lustre: 268863:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8392.181397] Lustre: lustre-OST0001: Aborting recovery [ 8397.330117] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8397.371438] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8397.417077] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8397.464994] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8397.469333] LustreError: 268638:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8399.395154] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8399.406955] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8402.129281] LustreError: 268605:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 181us: rc = -2 [ 8402.659290] LustreError: 268605:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1121us: rc = -2 [ 8402.687586] LustreError: 268605:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 8419.808835] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8419.812398] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8419.813663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8419.854966] Lustre: Skipped 3 previous similar messages [ 8422.042847] Lustre: server umount lustre-MDT0000 complete [ 8423.115964] LustreError: 268590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236498 with bad export cookie 15114238777964351067 [ 8423.128410] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8424.929922] LustreError: 268602:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8424.933515] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8424.935574] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8424.935582] Lustre: Skipped 3 previous similar messages [ 8424.956440] LustreError: 268602:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8424.992616] Lustre: Skipped 1 previous similar message [ 8428.959965] LustreError: 268604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8428.960521] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8428.974392] LustreError: 268604:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8428.986104] Lustre: Skipped 2 previous similar messages [ 8429.827670] Lustre: server umount lustre-MDT0001 complete [ 8445.408448] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8445.556390] Lustre: server umount lustre-OST0000 complete [ 8452.668890] Lustre: server umount lustre-OST0001 complete [ 8461.178184] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:55:34 (1781236534) [ 8499.040114] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 8510.432800] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 8545.713732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8546.104361] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8546.134408] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8546.199728] Lustre: lustre-MDT0000: new disk, initializing [ 8546.341484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8546.346878] Lustre: Skipped 1 previous similar message [ 8546.382721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8550.592450] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8562.347441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8562.462971] Lustre: 272901:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8562.521992] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8562.525967] Lustre: Skipped 1 previous similar message [ 8562.549941] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 8562.603498] Lustre: lustre-MDT0001: new disk, initializing [ 8562.687246] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8562.718051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8562.729200] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8567.133343] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8575.274892] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8579.196302] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8585.976236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8586.276409] Lustre: lustre-OST0000: new disk, initializing [ 8586.280452] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8586.286198] Lustre: 274642:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8586.397173] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8592.832880] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8595.529786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8595.544162] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8595.697879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8603.168670] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8609.316148] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8622.899536] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8627.158957] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8632.072730] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8682.976715] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8682.980714] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8682.998885] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8683.019405] Lustre: Skipped 1 previous similar message [ 8684.284520] Lustre: server umount lustre-OST0000 complete [ 8690.656471] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8690.671622] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8693.221149] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8693.229645] Lustre: Skipped 3 previous similar messages [ 8695.570180] Lustre: server umount lustre-MDT0000 complete [ 8698.340280] LustreError: 272910:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8698.354821] LustreError: 272910:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8700.224460] LustreError: 275384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236775 with bad export cookie 15114238777964353783 [ 8700.224945] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8700.238090] LustreError: 275384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8700.638419] Lustre: server umount lustre-MDT0001 complete [ 8710.022948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8715.695700] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8721.090533] Lustre: server umount MGS complete [ 8734.669547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8735.322250] LustreError: 278339:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8735.379889] LustreError: 278352:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8735.467371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8740.163977] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8740.833760] LustreError: 278340:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8749.078547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8754.207637] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8760.769658] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8764.904576] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8771.442186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8771.728547] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8771.736732] Lustre: Skipped 1 previous similar message [ 8778.359935] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8784.083884] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8816.095945] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8816.100340] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8816.104961] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8816.107362] Lustre: Skipped 2 previous similar messages [ 8821.231961] LustreError: 279937:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8821.253632] LustreError: 279937:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8821.466811] Lustre: server umount lustre-OST0000 complete [ 8826.336792] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8826.342356] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8826.374733] LustreError: Skipped 1 previous similar message [ 8831.533675] Lustre: server umount lustre-MDT0000 complete [ 8835.985197] LustreError: 280852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236911 with bad export cookie 15114238777964355722 [ 8836.001847] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8836.014993] LustreError: 280852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8836.050281] LustreError: 266227:0:(import.c:719:ptlrpc_connect_import_locked()) already connecting [ 8836.410665] Lustre: server umount lustre-MDT0001 complete [ 8848.134060] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 8851.183484] Key type lgssc unregistered [ 8851.451426] LNet: 281943:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8851.457428] LNetError: 281943:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8851.469719] LNet: Removed LNI 192.168.201.102@tcp [ 8852.214860] Key type .llcrypt unregistered [ 8852.219773] Key type ._llcrypt unregistered [ 8895.097771] Key type ._llcrypt registered [ 8895.099854] Key type .llcrypt registered [ 8895.227935] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 8911.187366] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 8912.276275] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8912.432768] alg: No test for adler32 (adler32-zlib) [ 8913.494806] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 8913.854124] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 8915.661351] Key type lgssc registered [ 8917.441285] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8951.443481] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8951.476524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8952.825244] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8952.850873] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8952.955883] Lustre: lustre-MDT0000: new disk, initializing [ 8953.058265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8953.075779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8958.292590] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8969.025098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8969.172779] Lustre: 285867:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8969.250890] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8969.256606] Lustre: Skipped 1 previous similar message [ 8969.331935] Lustre: lustre-MDT0001: new disk, initializing [ 8969.435053] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8969.485644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8969.499273] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8974.319546] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 8983.095436] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8988.122507] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8995.192782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8995.400971] Lustre: lustre-OST0000: new disk, initializing [ 8995.404847] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8995.409942] Lustre: 287607:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8995.493834] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9001.686807] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 9004.137397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9004.148768] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9004.283822] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9008.854845] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9015.942104] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9026.293560] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9036.634547] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9041.219977] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9086.432080] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9086.443123] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9086.451460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9086.471528] Lustre: Skipped 1 previous similar message [ 9089.803345] Lustre: server umount lustre-OST0000 complete [ 9096.676894] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9096.702688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9096.713866] Lustre: Skipped 2 previous similar messages [ 9100.199583] Lustre: server umount lustre-MDT0000 complete [ 9104.393916] LustreError: 285859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237179 with bad export cookie 10784496040561298402 [ 9104.395845] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9104.415037] LustreError: 285859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9104.836748] Lustre: server umount lustre-MDT0001 complete [ 9116.043744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9122.718470] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 9128.801176] Lustre: server umount MGS complete [ 9142.275342] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9142.881546] LustreError: 291396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9142.898982] LustreError: 291396:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9142.914864] LustreError: 291409:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9143.031090] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9148.388993] LustreError: 291397:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9148.429875] LustreError: 291397:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9149.003567] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 9153.512385] LustreError: 291396:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9158.629707] LustreError: 291397:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9158.808881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9159.296375] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9163.656673] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 9168.671809] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9172.211733] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9179.145727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9179.477574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9186.929912] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 9192.340112] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9224.675677] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9224.701097] Lustre: Skipped 1 previous similar message [ 9224.708267] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9224.708656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9224.721778] LustreError: Skipped 1 previous similar message [ 9224.743872] Lustre: Skipped 1 previous similar message [ 9229.076518] Lustre: server umount lustre-OST0000 complete [ 9234.912863] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9234.921439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9234.926103] Lustre: Skipped 2 previous similar messages [ 9234.941030] Lustre: Skipped 1 previous similar message [ 9239.819139] Lustre: server umount lustre-MDT0000 complete [ 9240.035832] LustreError: 292416:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9240.049634] LustreError: 292416:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9243.558830] LustreError: 291377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237318 with bad export cookie 10784496040561300243 [ 9243.561715] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9243.570760] LustreError: 291377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9244.090741] Lustre: server umount lustre-MDT0001 complete [ 9259.236947] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [ 9263.548366] Key type lgssc unregistered [ 9264.058905] LNet: 294986:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9264.066540] LNetError: 294986:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9264.100068] LNet: Removed LNI 192.168.201.102@tcp [ 9265.606463] Key type .llcrypt unregistered [ 9265.608312] Key type ._llcrypt unregistered [ 9288.607500] Key type ._llcrypt registered [ 9288.609667] Key type .llcrypt registered [ 9288.690973] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [ 9302.193643] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 9303.044343] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9303.103892] alg: No test for adler32 (adler32-zlib) [ 9304.134658] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [ 9304.330067] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [ 9305.967216] Key type lgssc registered [ 9307.117221] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9329.713566] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [ 9344.068833] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9344.094098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9345.373669] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9345.406777] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9345.509283] Lustre: lustre-MDT0000: new disk, initializing [ 9345.653866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9345.683906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9349.926540] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [ 9359.504048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9359.720476] Lustre: lustre-OST0000: new disk, initializing [ 9359.727818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9359.730801] Lustre: Skipped 1 previous similar message [ 9359.735379] Lustre: 298968:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9359.786380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9359.951117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9359.961160] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9359.989350] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9366.365805] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10130.913333] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10130.925903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10132.483866] Lustre: server umount lustre-MDT0000 complete [10136.544714] LustreError: 300007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238211 with bad export cookie 14885567565220705620 [10136.548258] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10136.554154] LustreError: 300007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10136.718488] Lustre: server umount lustre-OST0000 complete [10165.503645] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [10176.769934] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [10215.280818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10215.442575] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10215.466231] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10215.526382] Lustre: lustre-MDT0000: new disk, initializing [10215.585920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10215.595361] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10219.874847] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10230.829728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10231.074963] Lustre: 303246:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10231.149770] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10231.154645] Lustre: Skipped 1 previous similar message [10231.329372] Lustre: lustre-MDT0001: new disk, initializing [10231.514728] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10231.554290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10231.569782] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10235.945722] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10243.428246] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10247.063496] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10254.818963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10255.246518] Lustre: lustre-OST0000: new disk, initializing [10255.252593] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10255.260705] Lustre: 304987:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10255.398588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10261.535916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10261.548052] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10261.708483] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10262.345773] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10269.564333] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10274.737930] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10274.993257] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10279.562618] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10279.686623] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10282.468661] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10282.470240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10282.477952] LustreError: Skipped 1 previous similar message [10282.479692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10282.498047] Lustre: Skipped 1 previous similar message [10282.514619] Lustre: Skipped 1 previous similar message [10287.585560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10287.592182] Lustre: Skipped 1 previous similar message [10292.707302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10292.714282] Lustre: Skipped 1 previous similar message [10296.288284] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10296.553447] Lustre: server umount lustre-OST0000 complete [10302.945207] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10302.963554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10302.981459] Lustre: Skipped 1 previous similar message [10302.991592] Lustre: Skipped 1 previous similar message [10307.029793] Lustre: server umount lustre-MDT0000 complete [10310.970657] LustreError: 303239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238386 with bad export cookie 14885567565223646439 [10310.978076] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10310.982291] LustreError: 303239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10311.385055] Lustre: server umount lustre-MDT0001 complete [10314.831457] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10316.627916] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 00:26:30 (1781238390) [10322.992393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10323.434889] LustreError: 307154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10323.457921] LustreError: 307154:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10323.524958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10328.551813] LustreError: 307155:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10329.130480] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10333.671462] LustreError: 307154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10338.792400] LustreError: 307155:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10340.530879] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10340.909898] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10345.100652] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10350.836410] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10354.726539] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10361.255152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10361.651665] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10368.567652] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10374.629877] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10381.096418] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10381.318846] Lustre: lustre-OST0001: new disk, initializing [10381.325670] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10381.336711] Lustre: 309651:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10381.460605] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10388.070170] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10389.140604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10389.152234] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10389.209343] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10389.220218] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10390.247555] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10391.268783] LustreError: 310056:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10391.282648] LustreError: 310056:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10393.316364] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10393.344883] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10396.354179] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10396.509531] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10397.481157] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10397.487977] LustreError: 310046:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10400.179692] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff891f817ae000.ost_server_uuid 50 [10401.884680] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff891f817ae000.ost_server_uuid in FULL state after 0 sec [10405.672656] LustreError: 310056:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10405.691418] LustreError: 310056:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10406.252370] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff891f817ae000.ost_server_uuid 50 [10407.754837] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff891f817ae000.ost_server_uuid in FULL state after 0 sec [10409.855434] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10423.637864] Lustre: lustre-OST0001: Not available for connect from 192.168.201.2@tcp (stopping) [10425.319772] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10425.331407] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10425.383294] Lustre: Skipped 1 previous similar message [10429.385617] Lustre: server umount lustre-OST0001 complete [10430.438163] LustreError: 308750:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10430.460495] LustreError: 308750:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10435.556920] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10435.567487] Lustre: Skipped 1 previous similar message [10440.675611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10440.684011] LustreError: 308748:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10440.688698] Lustre: Skipped 6 previous similar messages [10440.712914] LustreError: 308748:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10440.888824] Lustre: server umount lustre-OST0000 complete [10445.794234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10445.814780] Lustre: Skipped 1 previous similar message [10450.636268] Lustre: server umount lustre-MDT0000 complete [10454.689760] LustreError: 309649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238529 with bad export cookie 14885567565223647321 [10454.695426] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10454.708533] LustreError: 309649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10455.049864] Lustre: server umount lustre-MDT0001 complete [10466.079333] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [10468.740706] Key type lgssc unregistered [10469.002511] LNet: 312101:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10469.011647] LNetError: 312101:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10469.034967] LNet: Removed LNI 192.168.201.102@tcp [10469.792208] Key type .llcrypt unregistered [10469.794313] Key type ._llcrypt unregistered [10493.532925] Key type ._llcrypt registered [10493.535544] Key type .llcrypt registered [10493.665447] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [10494.589046] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10494.766158] alg: No test for adler32 (adler32-zlib) [10495.768497] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [10495.964295] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [10497.656259] Key type lgssc registered [10498.705748] Lustre: Echo OBD driver; http://www.lustre.org/ [10506.420520] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10506.448902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10508.138368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10512.815660] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10518.492558] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10518.499505] LustreError: 313521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10518.522744] LustreError: 313521:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10523.622378] LustreError: 313520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10525.645765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10526.197043] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10530.844440] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10540.970551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10541.492936] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10547.204080] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10551.728884] LustreError: 315067:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10553.057725] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10553.770471] LustreError: 315069:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10553.798719] LustreError: 315069:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10553.800223] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10558.955341] LustreError: 315070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10558.982712] LustreError: 315070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10559.258822] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10559.477522] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10564.608578] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10565.179889] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10570.624097] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10576.601812] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10576.981373] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10677.231827] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10677.242962] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10677.257570] LustreError: Skipped 1 previous similar message [10677.270928] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10682.338264] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10682.350901] Lustre: Skipped 2 previous similar messages [10683.006199] Lustre: server umount lustre-OST0001 complete [10687.465150] LustreError: 315069:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10687.495536] LustreError: 315069:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10692.066141] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10692.069213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10692.085258] Lustre: Skipped 1 previous similar message [10692.092638] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10696.861980] Lustre: server umount lustre-OST0000 complete [10700.257307] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10700.266773] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10700.277623] Lustre: Skipped 1 previous similar message [10700.282375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10700.285740] Lustre: Skipped 1 previous similar message [10702.816716] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10706.736444] Lustre: server umount lustre-MDT0000 complete [10707.938304] LustreError: 314590:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10707.954550] LustreError: 314590:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10710.819581] LustreError: 315975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238785 with bad export cookie 8427922701145746394 [10710.820917] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10710.828536] LustreError: 315975:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10711.198491] Lustre: server umount lustre-MDT0001 complete [10722.557297] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [10725.188676] Key type lgssc unregistered [10725.479395] LNet: 318070:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10725.493047] LNetError: 318070:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10725.511880] LNet: Removed LNI 192.168.201.102@tcp [10726.478177] Key type .llcrypt unregistered [10726.485703] Key type ._llcrypt unregistered [10770.688498] Key type ._llcrypt registered [10770.689808] Key type .llcrypt registered [10770.862902] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [10772.696195] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10772.805846] alg: No test for adler32 (adler32-zlib) [10774.054400] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [10774.326408] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [10776.023179] Key type lgssc registered [10777.720816] Lustre: Echo OBD driver; http://www.lustre.org/ [10789.978490] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10790.008199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10791.507950] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10796.002741] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10801.645027] LustreError: 320343:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10801.666983] LustreError: 320343:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10805.809410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10806.173579] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10811.737322] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10814.976391] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10822.296191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10822.943242] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10828.721654] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10831.143741] LustreError: 321802:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10836.285460] LustreError: 321804:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10836.312915] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10836.315225] LustreError: 321804:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10838.024806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10838.360949] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10843.636079] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10845.222810] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10853.087941] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10857.427965] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10869.219681] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10869.228932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10874.353499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10874.369218] Lustre: Skipped 6 previous similar messages [10874.930262] Lustre: server umount lustre-MDT0000 complete [10878.801740] LustreError: 320324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238953 with bad export cookie 18200648729241054151 [10878.808038] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10878.819619] LustreError: 320324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10879.368598] Lustre: server umount lustre-MDT0001 complete [10894.377506] Lustre: server umount lustre-OST0000 complete [10895.327421] Lustre: 319608:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781238954/real 1781238954] req@ffff8f99d52ea580 x1867764307390336/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781238970 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10895.343893] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10895.357606] Lustre: Skipped 3 previous similar messages [10898.327983] Lustre: server umount lustre-OST0001 complete [10913.740339] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [10923.751631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10924.129260] LustreError: 325528:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10924.242186] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10928.244024] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10929.635772] LustreError: 325529:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10936.595074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10937.080638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10941.184668] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10944.001788] Lustre: 326633:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10951.787632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10958.502050] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10962.416687] LustreError: 326988:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10962.449670] LustreError: 326988:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10967.255693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10967.508333] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10967.511250] Lustre: Skipped 1 previous similar message [10969.603800] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10969.613534] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10973.813369] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [10981.075291] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10984.909209] Lustre: 328470:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10990.055668] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10990.066818] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10990.068279] Lustre: Skipped 2 previous similar messages [10990.081762] Lustre: Skipped 2 previous similar messages [10995.171819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10995.188406] Lustre: Skipped 3 previous similar messages [10995.852528] Lustre: server umount lustre-MDT0000 complete [10999.030288] LustreError: 327010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781239074 with bad export cookie 18200648729241055852 [10999.039521] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10999.040211] LustreError: 327010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10999.555225] Lustre: server umount lustre-MDT0001 complete [11014.481175] Lustre: server umount lustre-OST0000 complete [11016.671162] Lustre: 319607:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781239075/real 1781239075] req@ffff8f99cc6a7480 x1867764307438080/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781239091 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11016.708975] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11016.725067] Lustre: Skipped 1 previous similar message [11018.098618] Lustre: server umount lustre-OST0001 complete [11021.691878] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11023.324975] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11025.539341] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 00:38:18 (1781239098) [11049.726522] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [11058.830389] loop0: detected capacity change from 0 to 1099511627776 [11059.398693] SGI XFS with ACLs, security attributes, quota, no debug enabled [11059.403822] *** VALIDATE xfs *** [11059.412384] XFS (loop0): Mounting V5 Filesystem [11059.434415] XFS (loop0): Ending clean mount [11060.844724] loop1: detected capacity change from 0 to 18691697672192 [11162.932655] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [11173.394745] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [11210.675567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11210.994493] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11211.041830] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11211.122412] Lustre: lustre-MDT0000: new disk, initializing [11211.257733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11211.313381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11216.014435] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11226.789488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11226.914955] Lustre: 333555:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11226.964242] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11226.968246] Lustre: Skipped 1 previous similar message [11227.025323] Lustre: lustre-MDT0001: new disk, initializing [11227.144824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11227.158935] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11231.523552] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11239.092879] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11243.177409] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11251.751567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11252.068114] Lustre: lustre-OST0000: new disk, initializing [11252.072545] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11252.087142] Lustre: 335297:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11254.034617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11254.051614] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11254.241963] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11258.205264] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11266.044542] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11270.729838] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11271.015614] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11276.050248] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11276.378868] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11279.850548] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11279.853167] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11279.855776] LustreError: Skipped 1 previous similar message [11279.859673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11279.880485] Lustre: Skipped 1 previous similar message [11284.632606] Lustre: server umount lustre-OST0000 complete [11290.080595] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11290.086143] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11290.098634] Lustre: Skipped 1 previous similar message [11290.108833] Lustre: Skipped 2 previous similar messages [11302.879679] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11303.179435] Lustre: server umount lustre-MDT0000 complete [11305.445982] LustreError: 333565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11305.467817] LustreError: 333565:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11306.463982] LustreError: 333545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781239381 with bad export cookie 18200648729241057406 [11306.468819] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11306.473550] LustreError: 333545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11306.787818] Lustre: server umount lustre-MDT0001 complete [11314.374686] XFS (loop0): Unmounting Filesystem [11319.743888] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 00:43:13 (1781239393) [11328.703951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11329.764906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11329.780696] Lustre: Skipped 2 previous similar messages [11334.719983] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11339.236317] LustreError: 337708:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11339.251626] LustreError: 337708:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11343.420654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11348.312946] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11355.403102] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11360.577891] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11368.901978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11375.987441] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11381.456190] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11399.137389] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11399.142503] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11399.148142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11399.148158] Lustre: Skipped 4 previous similar messages [11404.765700] Lustre: server umount lustre-OST0000 complete [11414.496609] LustreError: 338419:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11414.573327] Lustre: server umount lustre-MDT0000 complete [11418.361762] LustreError: 340269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781239493 with bad export cookie 18200648729241058288 [11418.363866] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11418.372565] LustreError: 340269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11418.857677] Lustre: server umount lustre-MDT0001 complete [11432.118774] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [11435.841294] Key type lgssc unregistered [11436.251629] LNet: 341003:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11436.261258] LNetError: 341003:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11436.292422] LNet: Removed LNI 192.168.201.102@tcp [11437.431328] Key type .llcrypt unregistered [11437.433192] Key type ._llcrypt unregistered [11472.610302] Key type ._llcrypt registered [11472.613879] Key type .llcrypt registered [11472.829983] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [11474.205413] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11474.342209] alg: No test for adler32 (adler32-zlib) [11475.661573] Lustre: Lustre: Build Version: 2.17.53_64_gab2e0dd [11475.955174] LNet: Added LNI 192.168.201.102@tcp [8/256/0/180] [11477.663821] Key type lgssc registered [11479.507821] Lustre: Echo OBD driver; http://www.lustre.org/ [11489.120591] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11489.136517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11490.787058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11495.206958] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11500.002839] LustreError: 342556:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11500.046753] LustreError: 342556:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11504.049639] LustreError: 342557:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11504.466635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11504.973896] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11509.814124] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11516.095567] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11520.277677] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11529.393695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11529.867469] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11537.003915] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11544.371064] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11555.808982] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11555.810324] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11555.825877] Lustre: Skipped 1 previous similar message [11555.847115] Lustre: Skipped 1 previous similar message [11557.213271] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.2@tcp (stopping) [11559.434883] Lustre: server umount lustre-MDT0001 complete [11560.933480] LustreError: 342551:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11560.942292] LustreError: 342551:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11566.065082] LustreError: 342557:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11566.097046] LustreError: 342557:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11569.763587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11569.892971] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11569.896427] Lustre: Found index 1 for lustre-MDT0001, updating log [11569.943837] Lustre: 342546:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11570.288879] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11575.441694] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11575.792514] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11577.884034] LustreError: 345750:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11577.903153] LustreError: 345750:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11577.927031] LustreError: 342549:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11578.029793] LustreError: 345758:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.102@tcp: cfg command failed: rc = -17 [11578.050054] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11578.050054] [11882.980961] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11882.990246] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11883.003610] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11888.105581] LustreError: 343531:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11888.121837] LustreError: 343531:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11888.146358] Lustre: server umount lustre-MDT0001 complete [11898.345916] LustreError: 342557:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11898.358127] LustreError: 342557:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [11899.306108] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11899.460857] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11899.467169] Lustre: Found index 1 for lustre-MDT0001, updating log [11899.544093] Lustre: 342545:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11899.968988] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11904.883963] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [11905.001484] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11907.083082] LustreError: 353755:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11907.099875] LustreError: 353755:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11907.127955] LustreError: 342549:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11907.139103] LustreError: 342549:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11907.244635] LustreError: 353761:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11907.259447] LustreError: 353761:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11907.277082] LustreError: 353761:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.102@tcp: cfg command failed: rc = -17 [11907.314028] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.102@tcp [11907.314028] [12212.199637] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12212.206974] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12212.228732] Lustre: Skipped 1 previous similar message [12212.243571] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12212.246398] Lustre: Skipped 2 previous similar messages [12215.272550] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12216.854819] Lustre: server umount lustre-MDT0001 complete [12220.403932] LustreError: 349692:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12220.427816] LustreError: 349692:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12228.954669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12229.108072] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12229.112331] Lustre: Found index 1 for lustre-MDT0001, updating log [12229.175026] Lustre: 342546:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12229.563545] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12234.431978] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [12234.733856] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12236.287764] LustreError: 361988:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12236.301634] LustreError: 361988:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12236.315622] LustreError: 342549:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12236.326479] LustreError: 342549:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12236.424225] LustreError: 361997:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12236.442976] LustreError: 361997:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.102@tcp: cfg command failed: rc = -114 [12236.457278] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12236.457278] [12557.288898] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12557.290406] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12557.292837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12557.292842] Lustre: Skipped 3 previous similar messages [12557.333706] Lustre: Skipped 1 previous similar message [12562.640276] Lustre: server umount lustre-MDT0000 complete [12566.607568] LustreError: 342537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240641 with bad export cookie 9823291431268603398 [12566.608327] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12566.614454] LustreError: 342537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12567.520915] LustreError: 342553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12567.525320] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12567.529284] LustreError: 342553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [12567.547840] Lustre: Skipped 1 previous similar message [12567.551869] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12567.556152] Lustre: Skipped 5 previous similar messages [12573.071994] Lustre: server umount lustre-MDT0001 complete [12576.845719] Lustre: server umount lustre-OST0000 complete [12585.862098] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [12593.497333] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [12625.989510] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:05:00 (1781240700) [12632.141337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12632.440892] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12632.470339] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12632.601522] Lustre: lustre-MDT0000: new disk, initializing [12632.721238] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12632.752936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12636.499855] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [12645.714798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12645.875812] Lustre: 372824:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12645.924970] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12645.927959] Lustre: Skipped 1 previous similar message [12645.984975] Lustre: lustre-MDT0001: new disk, initializing [12646.087985] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12646.131525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12646.141458] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12650.370635] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [12657.049751] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12660.754859] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12666.897833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12667.156374] Lustre: lustre-OST0000: new disk, initializing [12667.159837] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12667.168554] Lustre: 374564:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12667.279782] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12672.608492] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_default_debug -1 all [12674.095271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12674.103756] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12674.182459] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12679.334409] Lustre: DEBUG MARKER: oleg102-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12687.332690] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12687.335047] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12687.348750] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12687.356393] Lustre: Skipped 1 previous similar message [12692.050950] Lustre: server umount lustre-MDT0000 complete [12694.499441] LustreError: 374572:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12694.528311] LustreError: 374572:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12696.006666] LustreError: 372815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781240771 with bad export cookie 9823291431268605351 [12696.013740] LustreError: MGC192.168.201.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12696.021994] LustreError: 372815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12696.628477] Lustre: server umount lustre-MDT0001 complete [12712.029810] Lustre: server umount lustre-OST0000 complete [12744.304650] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing set_hostid [12754.011517] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing load_modules_local [12783.493616] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12500 sec ========= 01:07:37 (1781240857) [12784.905514] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 01:07:38 (1781240858) === [12788.753493] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 01:07:42 (1781240862) === [12825.659444] Lustre: DEBUG MARKER: oleg102-server.virtnet: executing unload_modules_local [12828.341331] Key type lgssc unregistered [12828.560350] LNet: 380766:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12828.566734] LNetError: 380766:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12829.606466] LNet: Removed LNI 192.168.201.102@tcp [12830.352151] Key type .llcrypt unregistered [12830.353597] Key type ._llcrypt unregistered