[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 514035908 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2480MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.003139] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008459] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010013] pid_max: default: 32768 minimum: 301 [ 0.011173] LSM: Security Framework initializing [ 0.013016] Yama: becoming mindful. [ 0.014037] SELinux: Initializing. [ 0.015067] *** VALIDATE selinux *** [ 0.023684] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028395] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030061] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031117] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032125] *** VALIDATE tmpfs *** [ 0.033486] *** VALIDATE proc *** [ 0.035125] *** VALIDATE cgroup *** [ 0.036009] *** VALIDATE cgroup2 *** [ 0.037289] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039122] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042031] Spectre V2 : User space: Vulnerable [ 0.043011] Speculative Store Bypass: Vulnerable [ 0.045893] debug: unmapping init [mem 0xffffffff94259000-0xffffffff94260fff] [ 0.047179] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048726] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049025] ... version: 2 [ 0.050014] ... bit width: 48 [ 0.051011] ... generic registers: 4 [ 0.052012] ... value mask: 0000ffffffffffff [ 0.053014] ... max period: 00007fffffffffff [ 0.054014] ... fixed-purpose events: 3 [ 0.055011] ... event mask: 000000070000000f [ 0.056327] rcu: Hierarchical SRCU implementation. [ 0.058500] smp: Bringing up secondary CPUs ... [ 0.059570] x86: Booting SMP configuration: [ 0.060024] .... node #0, CPUs: #1 #2 #3 [ 0.065070] smp: Brought up 1 node, 4 CPUs [ 0.067026] smpboot: Max logical packages: 1 [ 0.068011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.149124] node 0 deferred pages initialised in 78ms [ 0.153170] devtmpfs: initialized [ 0.154242] x86/mm: Memory block size: 128MB [ 0.158709] gcov: version magic: 0x41383552 [ 0.164229] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.165073] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.168426] pinctrl core: initialized pinctrl subsystem [ 0.171204] [ 0.171764] ************************************************************* [ 0.175012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.177012] ** ** [ 0.180017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.183013] ** ** [ 0.186077] ** This means that this kernel is built to expose internal ** [ 0.188013] ** IOMMU data structures, which may compromise security on ** [ 0.191013] ** your system. ** [ 0.193010] ** ** [ 0.196018] ** If you see this message and you are not debugging the ** [ 0.198010] ** kernel, report this immediately to your vendor! ** [ 0.201012] ** ** [ 0.204014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.206016] ************************************************************* [ 0.209534] NET: Registered protocol family 16 [ 0.212967] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.216120] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.220326] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.224265] cpuidle: using governor menu [ 0.225656] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.229049] PCI: Using configuration type 1 for base access [ 0.233137] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.243505] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.246042] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.251081] cryptd: max_cpu_qlen set to 1000 [ 0.255283] ACPI: Added _OSI(Module Device) [ 0.257014] ACPI: Added _OSI(Processor Device) [ 0.259014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.260014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.266660] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.272521] ACPI: Interpreter enabled [ 0.274064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.276014] ACPI: Using IOAPIC for interrupt routing [ 0.278229] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.283395] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.294585] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.297073] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.300036] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.304145] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.309549] acpiphp: Slot [2] registered [ 0.311159] acpiphp: Slot [5] registered [ 0.313135] acpiphp: Slot [6] registered [ 0.315136] acpiphp: Slot [7] registered [ 0.316216] acpiphp: Slot [8] registered [ 0.318135] acpiphp: Slot [9] registered [ 0.320111] acpiphp: Slot [10] registered [ 0.321254] acpiphp: Slot [3] registered [ 0.323104] acpiphp: Slot [4] registered [ 0.325116] acpiphp: Slot [11] registered [ 0.326143] acpiphp: Slot [12] registered [ 0.328148] acpiphp: Slot [13] registered [ 0.330193] acpiphp: Slot [14] registered [ 0.332200] acpiphp: Slot [15] registered [ 0.333111] acpiphp: Slot [16] registered [ 0.335111] acpiphp: Slot [17] registered [ 0.337168] acpiphp: Slot [18] registered [ 0.338118] acpiphp: Slot [19] registered [ 0.340209] acpiphp: Slot [20] registered [ 0.342112] acpiphp: Slot [21] registered [ 0.343114] acpiphp: Slot [22] registered [ 0.345093] acpiphp: Slot [23] registered [ 0.346070] acpiphp: Slot [24] registered [ 0.347082] acpiphp: Slot [25] registered [ 0.349119] acpiphp: Slot [26] registered [ 0.350000] acpiphp: Slot [27] registered [ 0.351166] acpiphp: Slot [28] registered [ 0.352093] acpiphp: Slot [29] registered [ 0.354135] acpiphp: Slot [30] registered [ 0.356135] acpiphp: Slot [31] registered [ 0.358125] PCI host bridge to bus 0000:00 [ 0.359021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.362026] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.365116] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.368026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.371027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.374022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.376228] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.380298] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.383379] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.395015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.401876] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.406029] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.408017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.412025] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.415570] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.420386] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.424049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.426907] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.434016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.450017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.456911] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.462588] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.474022] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.484015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.501016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.515990] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.528038] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.536016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.555020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.567979] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.576014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.586017] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.604015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.618631] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.625022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.632016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.650022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.662941] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.670016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.677020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.702027] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.720010] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.729017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.737018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.759018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.774000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.776426] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.779647] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.784430] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.787498] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.793335] iommu: Default domain type: Passthrough [ 0.795433] SCSI subsystem initialized [ 0.797148] ACPI: bus type USB registered [ 0.799094] usbcore: registered new interface driver usbfs [ 0.801094] usbcore: registered new interface driver hub [ 0.803130] usbcore: registered new device driver usb [ 0.805219] pps_core: LinuxPPS API ver. 1 registered [ 0.807011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.811083] PTP clock support registered [ 0.814165] EDAC MC: Ver: 3.0.0 [ 0.816168] PCI: Using ACPI for IRQ routing [ 0.818297] NetLabel: Initializing [ 0.821012] NetLabel: domain hash size = 128 [ 0.822009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.825099] NetLabel: unlabeled traffic allowed by default [ 0.828108] vgaarb: loaded [ 0.830135] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.831011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.839833] clocksource: Switched to clocksource kvm-clock [ 0.957067] VFS: Disk quotas dquot_6.6.0 [ 0.959224] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.962889] *** VALIDATE ramfs *** [ 0.964447] *** VALIDATE hugetlbfs *** [ 0.966440] pnp: PnP ACPI init [ 0.970121] pnp: PnP ACPI: found 6 devices [ 0.988127] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.992070] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.995045] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.998288] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.000682] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.003233] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.006540] NET: Registered protocol family 2 [ 1.009471] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.014855] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.018815] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.024986] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.028389] TCP: Hash tables configured (established 65536 bind 65536) [ 1.031648] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.035311] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.038755] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.042797] NET: Registered protocol family 1 [ 1.045857] RPC: Registered named UNIX socket transport module. [ 1.048503] RPC: Registered udp transport module. [ 1.050388] RPC: Registered tcp transport module. [ 1.052428] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.055162] NET: Registered protocol family 44 [ 1.057611] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.059630] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.062094] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.064705] PCI: CLS 0 bytes, default 64 [ 1.066524] Unpacking initramfs... [ 2.488284] debug: unmapping init [mem 0xffff8ac63cc54000-0xffff8ac63ffbffff] [ 2.491835] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.493835] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.496402] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.015828] Initialise system trusted keyrings [ 3.017877] Key type blacklist registered [ 3.020178] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.029885] zbud: loaded [ 3.069300] *** VALIDATE nfs *** [ 3.070651] *** VALIDATE nfs4 *** [ 3.072672] pstore: using deflate compression [ 3.077292] Platform Keyring initialized [ 3.222449] NET: Registered protocol family 38 [ 3.224305] Key type asymmetric registered [ 3.225887] Asymmetric key parser 'x509' registered [ 3.227414] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.230145] io scheduler mq-deadline registered [ 3.232334] io scheduler kyber registered [ 3.234323] io scheduler bfq registered [ 3.236282] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.239768] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.242950] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.247745] ACPI: Power Button [PWRF] [ 3.253828] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.264145] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.286965] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.298813] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.316471] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.345538] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.375770] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.383541] Non-volatile memory driver v1.3 [ 3.385233] Linux agpgart interface v0.103 [ 3.419089] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.422440] vda: detected capacity change from 0 to 74698752 [ 3.438794] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.442153] vdb: detected capacity change from 0 to 1073741824 [ 3.460430] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.464044] vdc: detected capacity change from 0 to 2621440000 [ 3.484928] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.488640] vdd: detected capacity change from 0 to 2621440000 [ 3.505129] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.508348] vde: detected capacity change from 0 to 4294967296 [ 3.525138] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.528279] vdf: detected capacity change from 0 to 4294967296 [ 3.534739] libphy: Fixed MDIO Bus: probed [ 3.540824] usbcore: registered new interface driver usbserial_generic [ 3.543836] usbserial: USB Serial support registered for generic [ 3.546599] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.551238] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.552869] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.555789] mousedev: PS/2 mouse device common for all mice [ 3.559568] rtc_cmos 00:05: RTC can wake from S4 [ 3.563942] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.568737] rtc_cmos 00:05: registered as rtc0 [ 3.570879] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.573236] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.577125] intel_pstate: CPU model not supported [ 3.583887] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.587329] hid: raw HID events driver (C) Jiri Kosina [ 3.591874] usbcore: registered new interface driver usbhid [ 3.594900] usbhid: USB HID core driver [ 3.597271] drop_monitor: Initializing network drop monitor service [ 3.600833] Initializing XFRM netlink socket [ 3.603444] NET: Registered protocol family 10 [ 3.607702] Segment Routing with IPv6 [ 3.609996] NET: Registered protocol family 17 [ 3.612807] mpls_gso: MPLS GSO support [ 3.620735] RAS: Correctable Errors collector initialized. [ 3.622780] AVX version of gcm_enc/dec engaged. [ 3.624683] AES CTR mode by8 optimization enabled [ 3.711795] sched_clock: Marking stable (3711716909, 0)->(4589762742, -878045833) [ 3.715314] registered taskstats version 1 [ 3.717189] Loading compiled-in X.509 certificates [ 3.718800] zswap: loaded using pool lzo/zbud [ 3.746601] Key type big_key registered [ 3.762853] Key type encrypted registered [ 3.764897] ima: No TPM chip found, activating TPM-bypass! [ 3.767583] ima: Allocated hash algorithm: sha1 [ 3.772743] ima: No architecture policies found [ 3.774836] evm: Initialising EVM extended attributes: [ 3.777060] evm: security.selinux [ 3.778466] evm: security.ima [ 3.779474] evm: security.capability [ 3.780751] evm: HMAC attrs: 0x1 [ 3.784918] rtc_cmos 00:05: setting system clock to 2026-09-03 14:01:17 UTC (1788444077) [ 3.791253] debug: unmapping init [mem 0xffffffff95203000-0xffffffff953fffff] [ 3.795120] debug: unmapping init [mem 0xffffffff93f82000-0xffffffff94258fff] [ 3.804374] Write protecting the kernel read-only data: 28672k [ 3.808763] debug: unmapping init [mem 0xffffffff92603000-0xffffffff927fffff] [ 3.811917] debug: unmapping init [mem 0xffffffff92f14000-0xffffffff92ffffff] [ 3.850514] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.860903] systemd[1]: Detected virtualization kvm. [ 3.876555] systemd[1]: Detected architecture x86-64. [ 3.878865] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.924365] systemd[1]: No hostname configured. [ 3.926350] systemd[1]: Set hostname to . [ 3.930400] random: systemd: uninitialized urandom read (16 bytes read) [ 3.933338] systemd[1]: Initializing machine ID from random generator. [ 4.008907] random: ln: uninitialized urandom read (6 bytes read) [ 4.096143] random: systemd: uninitialized urandom read (16 bytes read) [ 4.100131] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.104682] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.109431] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.903705] device-mapper: uevent: version 1.0.3 [ 4.906338] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.784235] virtio_net virtio0 ens2: renamed from eth0 [ 5.815312] random: fast init done [ 5.837119] scsi host0: ata_piix [ 5.852256] scsi host1: ata_piix [ 5.855834] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.858957] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.602863] dracut-initqueue[591]: RTNETLINK answers: File exists [ 10.303158] random: crng init done [ 10.305699] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.936820] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.275754] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.545062] SELinux: Disabled at runtime. [ 12.607438] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.618204] systemd[1]: Detected virtualization kvm. [ 12.620261] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.142237] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.148500] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.153405] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.157567] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.161772] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.173988] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.191556] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ 13.246479] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Apply Kernel Variables... Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Paths. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Process Core Dump Socket. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.703641] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.929739] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.997195] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.155308] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.170245] EDAC sbridge: Ver: 1.1.2 [ 16.211350] Key type dns_resolver registered [ 17.090098] NFS: Registering the id_resolver key type [ 17.097281] Key type id_resolver registered [ 17.104087] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg423-server login: [ 51.579012] hrtimer: interrupt took 7724619 ns [ 89.805255] libcfs: loading out-of-tree module taints kernel. [ 89.837573] Key type ._llcrypt registered [ 89.839876] Key type .llcrypt registered [ 89.929486] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 111.067829] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 113.256801] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 113.276975] alg: No test for adler32 (adler32-zlib) [ 115.330207] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 116.528461] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 118.375234] Key type lgssc registered [ 120.200385] Lustre: Echo OBD driver; http://www.lustre.org/ [ 140.699491] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 186.070754] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 201.325738] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 201.364130] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 202.634434] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 202.654369] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 202.746754] Lustre: lustre-MDT0000: new disk, initializing [ 202.820769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 202.842675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 207.817865] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 222.293961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 222.439853] Lustre: 6521:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 222.528526] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 222.537841] Lustre: Skipped 1 previous similar message [ 222.647531] Lustre: lustre-MDT0001: new disk, initializing [ 222.740638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 222.793565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 222.807621] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 228.378477] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.997705] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 245.348801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 245.790930] Lustre: lustre-OST0000: new disk, initializing [ 245.793803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 245.798119] Lustre: 8459:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 245.849728] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 255.217543] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.047572] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 256.060255] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 256.180082] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 272.015900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 272.280770] Lustre: lustre-OST0001: new disk, initializing [ 272.289631] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 272.301531] Lustre: 9535:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 272.432148] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 279.390832] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 282.147555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 282.175297] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 282.271456] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 294.430423] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 303.781932] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 311.428982] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing check_logdir /tmp/testlogs/ [ 318.732420] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing yml_node [ 323.556786] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 326.035643] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 328.918120] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 330.559131] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 10:06:42 EDT 2026 [ 348.001288] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 349.766158] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 369.638944] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 369.640063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 369.653816] Lustre: Skipped 3 previous similar messages [ 369.665148] Lustre: Skipped 3 previous similar messages [ 373.070324] Lustre: server umount lustre-MDT0000 complete [ 377.949722] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444451 with bad export cookie 17047638993970716414 [ 377.952098] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 377.970420] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 379.872499] LustreError: 6526:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 379.874230] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 379.875854] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 379.891612] LustreError: 6526:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 379.931975] Lustre: Skipped 1 previous similar message [ 382.880507] LustreError: 6526:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 382.881042] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 382.908589] LustreError: 6526:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 382.922053] Lustre: Skipped 2 previous similar messages [ 384.406912] Lustre: server umount lustre-MDT0001 complete [ 388.381472] Lustre: server umount lustre-OST0000 complete [ 392.814663] Lustre: server umount lustre-OST0001 complete [ 401.041394] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 417.703254] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 461.100484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 461.396550] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 461.434768] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 461.564980] Lustre: lustre-MDT0000: new disk, initializing [ 461.663726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 461.684219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 466.803445] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 478.828631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.023403] Lustre: 15112:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 479.096413] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 479.108939] Lustre: Skipped 1 previous similar message [ 479.198655] Lustre: lustre-MDT0001: new disk, initializing [ 479.286804] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 479.336072] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 479.354791] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 484.445741] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 494.585164] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 500.804309] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 509.105819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 509.564415] Lustre: lustre-OST0000: new disk, initializing [ 509.574175] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 509.588614] Lustre: 17079:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 509.714896] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 511.343497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 511.364132] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 511.555492] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 516.176805] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 526.773468] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 532.215667] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 532.390396] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 536.912644] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 537.221756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 542.179067] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 542.181457] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 542.198580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 542.217975] Lustre: Skipped 1 previous similar message [ 545.927803] Lustre: server umount lustre-OST0000 complete [ 550.880252] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 550.890717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 550.907613] Lustre: Skipped 1 previous similar message [ 550.925232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 562.662659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 562.668550] Lustre: Skipped 5 previous similar messages [ 564.191274] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 564.473340] Lustre: server umount lustre-MDT0000 complete [ 567.777202] LustreError: 17788:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 567.806273] LustreError: 17788:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 568.318056] LustreError: 15102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444642 with bad export cookie 17047638993970717947 [ 568.319624] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 568.331850] LustreError: 15102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 568.878775] Lustre: server umount lustre-MDT0001 complete [ 575.315602] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:10:46 (1788444646) [ 583.809531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 584.438084] LustreError: 19347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 584.505803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 589.796829] LustreError: 19348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 590.643441] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 600.034957] LustreError: 19347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.047778] LustreError: 19347:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 602.667049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 611.100837] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 622.004131] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 628.626525] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 635.922639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 642.557607] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 650.266633] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 659.015339] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 669.665116] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 669.679128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 669.700661] Lustre: Skipped 1 previous similar message [ 669.711084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 672.559222] Lustre: server umount lustre-MDT0000 complete [ 675.298281] LustreError: 20389:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 675.335545] LustreError: 20389:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 676.913300] LustreError: 19329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444750 with bad export cookie 17047638993970718829 [ 676.918305] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 676.930357] LustreError: 19329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 677.398296] Lustre: server umount lustre-MDT0001 complete [ 691.926896] Lustre: server umount lustre-OST0000 complete [ 717.603931] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 728.974215] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 729.355341] LustreError: 23755:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 729.434893] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 729.445184] Lustre: Skipped 2 previous similar messages [ 735.268457] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 744.715985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 750.487485] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 754.017782] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 763.389409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 771.555839] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 782.511531] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 782.724521] Lustre: lustre-OST0001: new disk, initializing [ 782.727791] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 782.733672] Lustre: 26070:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 784.003466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 784.019659] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 784.086550] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 790.484300] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 801.781257] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 805.629783] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 820.212094] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 820.219759] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 820.244441] Lustre: Skipped 4 previous similar messages [ 820.260497] Lustre: Skipped 6 previous similar messages [ 825.327138] LustreError: 23755:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 825.353377] LustreError: 23755:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 825.419591] Lustre: server umount lustre-MDT0000 complete [ 829.226075] LustreError: 23737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444902 with bad export cookie 17047638993970719676 [ 829.233815] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 829.243818] LustreError: 23737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 829.802258] Lustre: server umount lustre-MDT0001 complete [ 844.571880] Lustre: server umount lustre-OST0000 complete [ 846.626283] Lustre: 3655:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788444904/real 1788444904] req@ffff8ac58238ed80 x1875319655230592/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788444920 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 846.647644] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 846.676446] Lustre: Skipped 1 previous similar message [ 848.397554] Lustre: server umount lustre-OST0001 complete [ 867.573453] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 879.868582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 880.263233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 880.271071] Lustre: Skipped 3 previous similar messages [ 886.048158] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 897.362200] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 904.019401] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 907.350203] Lustre: 30238:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 915.564054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 922.638633] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 932.143166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 939.766216] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 949.990439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 955.311964] Lustre: 32116:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 963.055818] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 963.069123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 963.088345] Lustre: Skipped 3 previous similar messages [ 963.108536] Lustre: Skipped 3 previous similar messages [ 968.545972] Lustre: server umount lustre-MDT0000 complete [ 973.022878] LustreError: 29068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445046 with bad export cookie 17047638993970720964 [ 973.038139] LustreError: 29068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 973.038444] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 973.290712] LustreError: 29085:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 973.322231] LustreError: 29085:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 973.715506] Lustre: server umount lustre-MDT0001 complete [ 980.700198] Lustre: server umount lustre-OST0000 complete [ 984.765126] Lustre: server umount lustre-OST0001 complete [ 995.079986] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:17:46 (1788445066) [ 1030.528841] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 1041.926080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1047.649484] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1056.802695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1063.053695] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1066.373272] Lustre: 36072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1074.208592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1081.734151] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1091.676232] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1099.472535] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1107.347223] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1111.419877] Lustre: 37950:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1117.671186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1117.689674] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1117.709347] Lustre: Skipped 5 previous similar messages [ 1117.741889] Lustre: Skipped 9 previous similar messages [ 1123.771840] Lustre: server umount lustre-MDT0000 complete [ 1127.869202] LustreError: 34901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445201 with bad export cookie 17047638993970722070 [ 1127.872642] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1127.885654] LustreError: 34901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1128.572891] Lustre: server umount lustre-MDT0001 complete [ 1135.632917] Lustre: server umount lustre-OST0000 complete [ 1140.865653] Lustre: server umount lustre-OST0001 complete [ 1150.243609] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:20:22 (1788445222) [ 1168.760587] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 1179.939441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1180.475136] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1180.481749] Lustre: Skipped 7 previous similar messages [ 1185.201652] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1194.388380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1199.503336] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1202.771623] Lustre: 41240:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1209.758587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1217.275905] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1227.308245] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1236.904933] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1244.914520] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1248.585047] Lustre: 43118:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1249.460693] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1249.467419] Lustre: Skipped 1 previous similar message [ 1253.361121] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1253.375498] Lustre: Skipped 4 previous similar messages [ 1255.511766] Lustre: server umount lustre-MDT0000 complete [ 1258.464817] LustreError: 40091:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1258.491795] LustreError: 40091:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 16 previous similar messages [ 1260.472494] LustreError: 40072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445334 with bad export cookie 17047638993970723183 [ 1260.474594] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1260.495219] LustreError: 40072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1266.989085] Lustre: server umount lustre-MDT0001 complete [ 1277.523846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1286.112113] LustreError: 3651:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8ac6bdba8000 x1875319655389696/t0(0) o250->MGC192.168.204.123@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1291.840788] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1296.889985] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1301.611319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1307.658961] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1307.676913] Lustre: Skipped 1 previous similar message [ 1308.178345] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1317.137209] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1319.500770] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1327.470521] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1329.574276] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1338.345129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1341.220897] Lustre: server umount lustre-MDT0000 complete [ 1344.940459] LustreError: 40072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445418 with bad export cookie 17047638993970724282 [ 1344.945394] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1344.955430] LustreError: 40072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1345.367331] Lustre: server umount lustre-MDT0001 complete [ 1360.038092] Lustre: server umount lustre-OST0000 complete [ 1374.536578] Lustre: server umount lustre-OST0001 complete [ 1386.511895] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:24:17 (1788445457) [ 1408.152840] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 1418.958543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1424.380398] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1433.629763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1439.139698] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1442.132802] Lustre: 48882:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1448.816492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1456.023762] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1466.238506] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1473.834234] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1483.231197] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1488.325916] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1502.705935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1502.711834] Lustre: Skipped 24 previous similar messages [ 1502.940114] Lustre: server umount lustre-MDT0000 complete [ 1507.278634] LustreError: 50765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445580 with bad export cookie 17047638993970725101 [ 1507.283405] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1507.298160] LustreError: 50765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1507.671771] Lustre: server umount lustre-MDT0001 complete [ 1521.883675] Lustre: server umount lustre-OST0000 complete [ 1524.197731] Lustre: 3652:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788445581/real 1788445581] req@ffff8ac585019180 x1875319655484928/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788445597 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1524.241600] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1524.254265] Lustre: Skipped 11 previous similar messages [ 1525.820426] Lustre: server umount lustre-OST0001 complete [ 1536.213901] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:26:48 (1788445608) [ 1539.187174] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1540.806973] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:26:52 (1788445612) [ 1554.597758] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 1566.341665] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 1593.554190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1593.855830] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1593.968356] Lustre: lustre-MDT0000: new disk, initializing [ 1594.101962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1599.383542] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1609.933880] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1617.990731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1618.333408] Lustre: lustre-OST0000: new disk, initializing [ 1618.337988] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1618.341978] Lustre: Skipped 1 previous similar message [ 1618.347627] Lustre: 54958:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1619.915310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1619.934525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1620.018090] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1626.196284] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1636.757801] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1639.071499] Lustre: DEBUG MARKER: create test files [ 1676.260658] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1676.665926] Lustre: server umount lustre-OST0000 complete [ 1682.960188] Lustre: server umount lustre-MDT0000 complete [ 1695.858316] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 1699.206483] Key type lgssc unregistered [ 1699.562839] LNet: 56599:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1699.579598] LNetError: 56599:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1699.597755] LNet: Removed LNI 192.168.204.123@tcp [ 1701.049188] Key type .llcrypt unregistered [ 1701.053170] Key type ._llcrypt unregistered [ 1730.877732] Key type ._llcrypt registered [ 1730.879362] Key type .llcrypt registered [ 1731.061772] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 1732.629594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1732.683344] alg: No test for adler32 (adler32-zlib) [ 1733.755315] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1734.042620] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 1735.719641] Key type lgssc registered [ 1737.043964] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1746.151784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1746.185855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1747.810367] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1753.210680] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1762.296572] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1769.863987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1770.167434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1773.261095] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1773.357931] Lustre: 59335:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1773.376173] Lustre: 59335:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1773.387630] Lustre: 59335:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1773.398830] Lustre: 59335:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1773.405790] Lustre: 59335:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1773.416246] Lustre: 59335:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1778.491534] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1787.070670] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1792.351370] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1798.117657] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1806.956755] Lustre: server umount lustre-OST0000 complete [ 1821.313734] Lustre: server umount lustre-MDT0000 complete [ 1833.611643] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 1836.774621] Key type lgssc unregistered [ 1837.086371] LNet: 60947:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1837.095538] LNetError: 60947:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1838.133719] LNet: Removed LNI 192.168.204.123@tcp [ 1839.327927] Key type .llcrypt unregistered [ 1839.330336] Key type ._llcrypt unregistered [ 1879.212537] Key type ._llcrypt registered [ 1879.214941] Key type .llcrypt registered [ 1879.320491] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 1880.943330] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1880.984250] alg: No test for adler32 (adler32-zlib) [ 1881.923849] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1882.214908] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 1883.975198] Key type lgssc registered [ 1885.483166] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1893.826179] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1893.856255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1895.381577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1899.703808] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1906.743771] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1913.197655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1913.517325] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1919.588320] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 1920.680415] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1920.789819] Lustre: 63779:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1920.807168] Lustre: 63779:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1920.815023] Lustre: 63779:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1920.824825] Lustre: 63779:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1920.832917] Lustre: 63779:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1920.843686] Lustre: 63779:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1926.847543] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1931.886681] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1941.474278] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1941.500917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1946.721967] Lustre: server umount lustre-OST0000 complete [ 1950.939718] Lustre: server umount lustre-MDT0000 complete [ 1961.296605] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 1964.141307] Key type lgssc unregistered [ 1964.397860] LNet: 65340:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1964.404656] LNetError: 65340:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1964.427335] LNet: Removed LNI 192.168.204.123@tcp [ 1965.264753] Key type .llcrypt unregistered [ 1965.270210] Key type ._llcrypt unregistered [ 1988.225741] Key type ._llcrypt registered [ 1988.234134] Key type .llcrypt registered [ 1988.379784] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 2009.434822] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 2010.884179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2010.980121] alg: No test for adler32 (adler32-zlib) [ 2012.250682] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 2012.546810] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 2014.191193] Key type lgssc registered [ 2015.384050] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2051.463685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2051.517847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2052.810176] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2052.866917] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2052.999404] Lustre: lustre-MDT0000: new disk, initializing [ 2053.127411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2053.147384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2058.862620] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2070.882599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2071.032218] Lustre: 68498:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2071.083180] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2071.089107] Lustre: Skipped 1 previous similar message [ 2071.195625] Lustre: lustre-MDT0001: new disk, initializing [ 2071.326188] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2071.366964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2071.388304] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2076.031824] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2084.240263] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2089.982193] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2097.354912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2097.590254] Lustre: lustre-OST0000: new disk, initializing [ 2097.593224] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2097.600991] Lustre: 70469:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2097.676818] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2103.370599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2103.398821] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2103.485776] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2104.820994] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2114.078645] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2118.540317] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2118.865499] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2122.637338] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2122.925898] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2129.377615] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2129.388818] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2129.399083] LustreError: Skipped 1 previous similar message [ 2129.405886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2129.430103] Lustre: Skipped 1 previous similar message [ 2129.450519] Lustre: Skipped 1 previous similar message [ 2131.158430] Lustre: server umount lustre-OST0000 complete [ 2138.083531] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2138.092247] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2138.108357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2139.619467] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2139.619781] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2139.645949] Lustre: Skipped 1 previous similar message [ 2144.740350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2144.749357] Lustre: Skipped 1 previous similar message [ 2149.343151] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2149.719692] Lustre: server umount lustre-MDT0000 complete [ 2153.442018] LustreError: 68490:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446227 with bad export cookie 16477929545588039248 [ 2153.449305] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2153.459665] LustreError: 68490:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2153.778879] Lustre: server umount lustre-MDT0001 complete [ 2162.486703] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:37:14 (1788446234) [ 2173.827693] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 2212.312580] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 2221.188780] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 2254.006888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2254.225542] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2254.254458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2254.337090] Lustre: lustre-MDT0000: new disk, initializing [ 2254.439642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2254.454506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2258.654611] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2268.250209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2268.354920] Lustre: 76487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2268.430588] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2268.440489] Lustre: Skipped 1 previous similar message [ 2268.531225] Lustre: lustre-MDT0001: new disk, initializing [ 2268.637778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2268.692407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2268.707583] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2272.988600] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2281.569824] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2286.886641] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2293.236463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2293.484943] Lustre: lustre-OST0000: new disk, initializing [ 2293.488212] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2293.495106] Lustre: 78456:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2293.584696] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2299.942886] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2301.059950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2301.076851] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2301.253116] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2309.099186] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2314.152237] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2314.413650] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2319.512868] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2319.802880] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2321.900134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2321.903309] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2321.911487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2321.932370] Lustre: Skipped 1 previous similar message [ 2332.134363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2332.139520] Lustre: Skipped 4 previous similar messages [ 2336.223856] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2336.450920] Lustre: server umount lustre-OST0000 complete [ 2340.322580] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2340.335192] LustreError: Skipped 1 previous similar message [ 2340.339785] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2346.253203] Lustre: server umount lustre-MDT0000 complete [ 2347.490232] LustreError: 76494:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2347.500950] LustreError: 76494:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2349.879013] LustreError: 76479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446423 with bad export cookie 16477929545588040123 [ 2349.882120] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2349.897612] LustreError: 76479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2350.235655] Lustre: server umount lustre-MDT0001 complete [ 2357.467882] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:40:29 (1788446429) [ 2363.061812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2363.504195] LustreError: 80866:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2363.580638] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2367.489640] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2368.999728] LustreError: 80867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2374.119584] LustreError: 80866:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2375.353187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2379.842350] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2386.102989] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2391.306332] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2398.217520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2398.745607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2398.751786] Lustre: Skipped 1 previous similar message [ 2405.497730] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2412.538864] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2416.210278] Lustre: 83538:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 2b10dc9f-f11f-4ba3-949d-68a66935a4ce at adminstrative request [ 2419.177343] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2419.190937] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2428.391505] Lustre: MGS: Client 2b10dc9f-f11f-4ba3-949d-68a66935a4ce (at 0@lo) reconnecting [ 2428.404254] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe4ad5d9e2eb06534 to 0xe4ad5d9e2eb067d4 [ 2428.417938] Lustre: MGC192.168.204.123@tcp: Connection restored to 0@lo (at 0@lo) [ 2453.200832] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2453.434752] Lustre: lustre-OST0001: new disk, initializing [ 2453.437395] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2453.442427] Lustre: 83946:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2459.721400] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2460.754214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2460.772768] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2460.922296] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2469.344816] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2476.518340] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2476.526868] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2476.538472] Lustre: Skipped 2 previous similar messages [ 2476.539495] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2476.550459] Lustre: Skipped 4 previous similar messages [ 2478.457432] Lustre: server umount lustre-OST0001 complete [ 2481.640055] LustreError: 82690:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2481.681821] LustreError: 82690:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2485.729788] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2485.755089] LustreError: Skipped 1 previous similar message [ 2489.434204] Lustre: server umount lustre-OST0000 complete [ 2496.992742] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2497.015688] Lustre: Skipped 2 previous similar messages [ 2508.255150] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2508.498250] Lustre: server umount lustre-MDT0000 complete [ 2512.150675] LustreError: 80849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446585 with bad export cookie 16477929545588041684 [ 2512.162838] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2512.560899] Lustre: server umount lustre-MDT0001 complete [ 2521.089613] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:43:13 (1788446593) [ 2522.310541] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2523.826866] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:43:15 (1788446595) [ 2525.353754] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2527.026869] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:43:18 (1788446598) [ 2528.445622] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2529.934742] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:43:22 (1788446602) [ 2531.201462] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2532.716790] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:43:24 (1788446604) [ 2534.020644] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2538.429426] loop: module loaded [ 2538.449255] loop0: detected capacity change from 0 to 4294967296 [ 2538.478584] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2539.666959] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2542.569251] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2545.620419] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2568.554133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2568.587983] LustreError: 88068:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2568.593942] LustreError: 88068:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2568.604098] LustreError: 88068:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2568.613126] LustreError: 88068:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2585.378150] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 2594.613208] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 2630.281516] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2630.538966] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2630.575571] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2630.654603] Lustre: lustre-MDT0000: new disk, initializing [ 2630.800313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2630.804923] Lustre: Skipped 1 previous similar message [ 2630.824179] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2635.274423] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2646.171568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2646.343938] Lustre: 90935:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2646.678294] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2650.737328] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2659.621329] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2664.923826] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2671.268881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2671.548802] Lustre: 92905:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2677.681537] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2680.880993] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2681.003079] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2686.571950] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2690.424346] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2690.643754] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2694.610896] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2694.823567] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2696.673217] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2696.678149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2696.687711] Lustre: Skipped 2 previous similar messages [ 2696.701128] Lustre: Skipped 10 previous similar messages [ 2702.700400] Lustre: server umount lustre-OST0000 complete [ 2712.819781] Lustre: server umount lustre-MDT0000 complete [ 2716.145422] LustreError: 90927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446789 with bad export cookie 16477929545588042293 [ 2716.153100] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2716.161668] LustreError: 90927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2716.590284] Lustre: server umount lustre-MDT0001 complete [ 2724.508661] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:46:36 (1788446796) [ 2731.515113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2732.155058] LustreError: 95317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2732.211237] LustreError: 95317:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2738.125926] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2745.945439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2750.321329] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2757.539648] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2762.765908] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2768.732974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2768.973786] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2768.981504] Lustre: Skipped 4 previous similar messages [ 2774.806278] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2781.931967] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2786.978939] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2787.198833] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2791.364537] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2791.572082] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2797.009246] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2797.111594] Lustre: lustre-OST0001: new disk, initializing [ 2797.115336] Lustre: Skipped 2 previous similar messages [ 2797.118713] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2797.123574] Lustre: Skipped 3 previous similar messages [ 2797.127554] Lustre: 98684:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2802.477985] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2805.799422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2805.812141] Lustre: Skipped 2 previous similar messages [ 2805.822329] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2805.897640] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2810.734886] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2814.916637] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2815.077982] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2819.091249] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2819.283545] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2837.331425] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2860.353123] Lustre: Failing over lustre-MDT0000 [ 2860.821399] Lustre: server umount lustre-MDT0000 complete [ 2862.049781] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2862.054791] LustreError: 99176:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2862.070509] Lustre: Skipped 5 previous similar messages [ 2862.091026] LustreError: 99176:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 2878.431440] Lustre: 66722:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788446935/real 1788446935] req@000000008d01f8aa x1875321643338496/t0(0) o400->MGC192.168.204.123@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788446951 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2878.446147] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2885.421893] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2885.429913] LDISKFS-fs (dm-0): recovery complete [ 2885.445185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2889.027740] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2894.100351] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2894.102169] Lustre: 95313:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2894.328427] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2894.386029] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2894.541396] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 2894.713483] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2894.733772] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2894.748739] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2895.075912] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2895.082569] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2895.401129] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2895.406757] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2895.744631] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2895.756784] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 2 previous similar messages [ 2896.079921] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2896.092665] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2896.415976] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2896.429435] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2898.050877] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2898.085063] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2898.097355] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2898.108171] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2898.423226] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2898.436557] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2902.249847] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2902.256442] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2902.261073] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2902.265805] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2902.583138] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2902.594482] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2910.550613] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2910.565756] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 2910.572771] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2910.584170] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2910.903135] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2910.911796] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2926.738246] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2926.761815] Lustre: 101530:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2926.771105] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2926.784550] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2927.103135] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2927.109221] LustreError: 101530:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2954.303431] Lustre: 101530:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2954.312717] Lustre: 101530:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2954.319746] Lustre: 101530:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 2bca2e20-a7e5-4724-a0cf-1222d48dee2b@192.168.204.23@tcp [ 2954.330738] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2954.337517] Lustre: 101530:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@000000008d42d63c x1875321621125888/t0(8589934785) o101->2bca2e20-a7e5-4724-a0cf-1222d48dee2b@192.168.204.23@tcp:18/0 lens 592/0 e 0 to 0 dl 1788447038 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2954.361811] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2954.376511] Lustre: lustre-MDT0000: Denying connection for new client 2bca2e20-a7e5-4724-a0cf-1222d48dee2b (at 192.168.204.23@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2954.397240] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2954.440620] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2954.448210] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2954.453215] Lustre: Skipped 2 previous similar messages [ 2954.507908] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 2954.511140] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 2970.593620] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2970.600103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2970.606430] Lustre: Skipped 6 previous similar messages [ 2975.431654] Lustre: server umount lustre-OST0000 complete [ 2976.230251] LustreError: 97137:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2976.251759] LustreError: 97137:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 35 previous similar messages [ 2984.899614] Lustre: server umount lustre-OST0001 complete [ 2990.568501] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2990.580779] Lustre: Skipped 4 previous similar messages [ 3002.335127] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3002.766374] Lustre: server umount lustre-MDT0000 complete [ 3006.382536] LustreError: 95298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447080 with bad export cookie 16477929545588093512 [ 3006.391908] LustreError: 95298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3006.393148] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3006.686178] Lustre: server umount lustre-MDT0001 complete [ 3014.162440] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:51:26 (1788447086) [ 3021.620748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3021.628597] Lustre: *** cfs_fail_loc=197, val=0*** [ 3037.215623] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3042.335567] LustreError: 103507:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.123@tcp: failed processing log, type 4: rc = -110 [ 3047.484844] Lustre: *** cfs_fail_loc=197, val=0*** [ 3047.489710] Lustre: Skipped 65 previous similar messages [ 3067.999305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3068.009478] Lustre: Skipped 2 previous similar messages [ 3074.050174] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3080.127068] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3081.804241] Lustre: server umount lustre-OST0000 complete [ 3089.878798] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:52:41 (1788447161) [ 3110.489686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3110.952679] LustreError: 105125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3110.976658] LustreError: 105125:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3115.108866] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3123.984251] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3128.104304] LustreError: 105156:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3128.117663] LustreError: 105156:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3128.123477] LustreError: 105156:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3128.610722] Lustre: server umount lustre-MDT0000 complete [ 3161.547614] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 3169.938217] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 3202.300694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3202.512934] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3202.546841] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3202.643899] Lustre: lustre-MDT0000: new disk, initializing [ 3202.711804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3206.691432] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3215.992476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3216.185658] Lustre: 109333:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3216.421951] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3220.449392] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3229.265457] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3234.986665] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3240.445650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3240.683864] Lustre: 111302:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3245.851301] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3247.308267] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3253.829203] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3258.060255] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3258.215074] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3262.114260] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3262.322580] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3268.064333] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3268.069606] LustreError: Skipped 4 previous similar messages [ 3268.071956] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3268.077190] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3268.087921] Lustre: Skipped 2 previous similar messages [ 3268.093578] Lustre: Skipped 13 previous similar messages [ 3278.303131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3278.459286] Lustre: server umount lustre-OST0000 complete [ 3288.192539] Lustre: server umount lustre-MDT0000 complete [ 3288.548638] LustreError: 110763:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3288.569646] LustreError: 110763:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3291.124857] LustreError: 109324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447364 with bad export cookie 16477929545588104257 [ 3291.139339] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3291.140846] LustreError: 109324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3291.420426] Lustre: server umount lustre-MDT0001 complete [ 3298.757906] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:56:10 (1788447370) [ 3306.023699] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 3308.484765] Key type lgssc unregistered [ 3308.733219] LNet: 113793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3308.742189] LNetError: 113793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3308.765715] LNet: Removed LNI 192.168.204.123@tcp [ 3309.425731] Key type .llcrypt unregistered [ 3309.428449] Key type ._llcrypt unregistered [ 3325.378687] Key type ._llcrypt registered [ 3325.386405] Key type .llcrypt registered [ 3325.501851] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 3338.305699] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 3339.512117] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3339.529291] alg: No test for adler32 (adler32-zlib) [ 3340.635405] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3340.945996] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 3342.624680] Key type lgssc registered [ 3343.405133] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3385.954755] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3385.977425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3387.283933] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3387.318378] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3387.413798] Lustre: lustre-MDT0000: new disk, initializing [ 3387.555672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3387.604057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3392.695918] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3403.554940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3403.887324] Lustre: lustre-OST0000: new disk, initializing [ 3403.900983] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3403.906142] Lustre: Skipped 1 previous similar message [ 3403.920149] Lustre: 117946:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3404.002513] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3404.626627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3404.640221] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3404.689291] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3410.072753] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3421.330047] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3421.611552] Lustre: lustre-OST0001: new disk, initializing [ 3421.615587] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3421.627184] Lustre: 118807:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3421.711456] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3428.606395] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3433.969949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3433.980788] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3434.021179] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3439.080448] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3439.083580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3439.098377] Lustre: Skipped 1 previous similar message [ 3443.611724] Lustre: server umount lustre-MDT0000 complete [ 3449.566667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3453.925682] LustreError: 117131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447527 with bad export cookie 14643406251396498165 [ 3453.928953] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3453.943094] LustreError: 117131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3454.020651] Lustre: server umount lustre-OST0000 complete [ 3458.036938] Lustre: server umount lustre-OST0001 complete [ 3466.107974] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:58:58 (1788447538) [ 3487.470830] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:59:19 (1788447559) [ 3500.171359] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 3519.875921] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:59:51 (1788447591) [ 3555.568938] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 3565.933968] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 3607.321140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3607.802714] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3607.878653] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3608.005881] Lustre: lustre-MDT0000: new disk, initializing [ 3608.204030] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3608.228925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3613.620848] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3624.254372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3624.377357] Lustre: 126575:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3624.447717] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3624.453240] Lustre: Skipped 1 previous similar message [ 3624.565253] Lustre: lustre-MDT0001: new disk, initializing [ 3624.667324] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3624.702456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3624.715459] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3629.284915] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3637.787495] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3643.292764] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3650.609724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3651.037567] Lustre: lustre-OST0000: new disk, initializing [ 3651.042264] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3651.048275] Lustre: 128545:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3651.135574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3657.708727] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3658.848077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3658.861474] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3658.997638] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3668.470456] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3673.763118] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3674.130493] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3679.813327] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3679.986489] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3684.840224] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3684.842545] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3684.844815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3684.844824] Lustre: Skipped 1 previous similar message [ 3684.860176] Lustre: Skipped 1 previous similar message [ 3684.886563] LustreError: Skipped 1 previous similar message [ 3688.371477] Lustre: server umount lustre-OST0000 complete [ 3695.083279] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3695.106141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3695.114880] Lustre: Skipped 2 previous similar messages [ 3698.815201] Lustre: server umount lustre-MDT0000 complete [ 3702.735795] LustreError: 126567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447776 with bad export cookie 14643406251396499376 [ 3702.740416] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3702.751247] LustreError: 126567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3703.015229] Lustre: server umount lustre-MDT0001 complete [ 3711.540908] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:03:03 (1788447783) [ 3717.778829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3718.499179] LustreError: 130957:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3718.530973] LustreError: 130957:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3718.595748] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3723.748941] LustreError: 130958:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3724.104697] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3728.866192] LustreError: 130957:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3733.985361] LustreError: 130958:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3734.027935] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3739.344786] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3747.665304] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3753.676269] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3759.855024] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3760.102793] LustreError: 132779:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3760.216640] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.225122] Lustre: Skipped 1 previous similar message [ 3765.543501] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3772.027409] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3776.082564] Lustre: *** cfs_fail_loc=159, val=0*** [ 3792.739381] Lustre: lustre-MDT0000: Client 319ff3e8-b816-4698-9a28-e3ae476aad79 (at 192.168.204.23@tcp) reconnecting [ 3794.087805] Lustre: *** cfs_fail_loc=159, val=0*** [ 3794.092238] Lustre: Skipped 5 previous similar messages [ 3810.615646] Lustre: lustre-MDT0000: Client 319ff3e8-b816-4698-9a28-e3ae476aad79 (at 192.168.204.23@tcp) reconnecting [ 3830.240871] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3830.244524] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3830.252518] LustreError: Skipped 1 previous similar message [ 3830.255647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3830.269204] Lustre: Skipped 2 previous similar messages [ 3830.300401] Lustre: Skipped 1 previous similar message [ 3833.454711] Lustre: server umount lustre-OST0000 complete [ 3840.484389] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3840.495990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3840.510052] Lustre: Skipped 1 previous similar message [ 3840.520747] Lustre: Skipped 1 previous similar message [ 3843.335256] Lustre: server umount lustre-MDT0000 complete [ 3845.600235] LustreError: 131686:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3845.620837] LustreError: 131686:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3847.301182] LustreError: 130937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447921 with bad export cookie 14643406251396500265 [ 3847.319989] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3847.851389] Lustre: server umount lustre-MDT0001 complete [ 3858.903549] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 3862.771776] Key type lgssc unregistered [ 3863.185260] LNet: 134737:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3863.196889] LNetError: 134737:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3863.232184] LNet: Removed LNI 192.168.204.123@tcp [ 3864.156765] Key type .llcrypt unregistered [ 3864.159928] Key type ._llcrypt unregistered [ 3891.859076] Key type ._llcrypt registered [ 3891.867704] Key type .llcrypt registered [ 3892.026542] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 3893.179940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3893.421976] alg: No test for adler32 (adler32-zlib) [ 3894.483640] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3894.712337] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 3896.415196] Key type lgssc registered [ 3897.234116] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3905.793599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3905.815078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3907.448050] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3912.683506] LustreError: 136151:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3912.711263] LustreError: 136151:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3913.152619] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3917.795976] LustreError: 136152:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3922.913607] LustreError: 136151:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3923.223127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3923.564865] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3927.331048] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3934.153282] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3938.809148] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3944.779164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3945.078382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3951.428645] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 3953.269554] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3959.197246] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3969.076637] Lustre: *** cfs_fail_loc=159, val=0*** [ 3984.703065] Lustre: lustre-MDT0000: Client 1da99664-dfc8-492e-937d-ba9045a6dcf1 (at 192.168.204.23@tcp) reconnecting [ 3986.860601] Lustre: *** cfs_fail_loc=159, val=0*** [ 3986.869631] Lustre: Skipped 2 previous similar messages [ 4003.212601] Lustre: lustre-MDT0001: Client 1da99664-dfc8-492e-937d-ba9045a6dcf1 (at 192.168.204.23@tcp) reconnecting [ 4004.774690] Lustre: *** cfs_fail_loc=159, val=0*** [ 4021.057892] Lustre: lustre-MDT0001: Client 1da99664-dfc8-492e-937d-ba9045a6dcf1 (at 192.168.204.23@tcp) reconnecting [ 4024.982408] Lustre: 139190:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4027.833415] Lustre: *** cfs_fail_loc=159, val=0*** [ 4027.850789] Lustre: Skipped 4 previous similar messages [ 4044.146069] Lustre: lustre-MDT0000: Client c62f4d03-38a4-4ab6-b522-814b99cf6cae (at 192.168.204.23@tcp) reconnecting [ 4045.745602] Lustre: *** cfs_fail_loc=159, val=0*** [ 4045.747432] Lustre: Skipped 13 previous similar messages [ 4062.006068] Lustre: lustre-MDT0000: Client c62f4d03-38a4-4ab6-b522-814b99cf6cae (at 192.168.204.23@tcp) reconnecting [ 4063.476407] Lustre: 139481:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4069.866495] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4069.867714] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4069.872053] LustreError: Skipped 1 previous similar message [ 4069.895371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4071.392432] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4071.399259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4071.407265] LustreError: Skipped 4 previous similar messages [ 4071.407311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4071.445307] Lustre: Skipped 1 previous similar message [ 4075.972460] Lustre: server umount lustre-OST0000 complete [ 4081.641498] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4081.675232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4081.690722] Lustre: Skipped 1 previous similar message [ 4086.128613] Lustre: server umount lustre-MDT0000 complete [ 4086.756517] LustreError: 136152:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4086.798188] LustreError: 136152:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4091.872936] LustreError: 139364:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4091.950672] LustreError: 139364:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4092.681875] LustreError: 136133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448166 with bad export cookie 8608711563152421815 [ 4092.682777] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4092.716940] LustreError: 136133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4093.612413] Lustre: server umount lustre-MDT0001 complete [ 4113.676888] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 4117.202977] Key type lgssc unregistered [ 4117.506053] LNet: 140558:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4117.514563] LNetError: 140558:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4117.532655] LNet: Removed LNI 192.168.204.123@tcp [ 4118.691662] Key type .llcrypt unregistered [ 4118.694420] Key type ._llcrypt unregistered [ 4143.550681] Key type ._llcrypt registered [ 4143.557275] Key type .llcrypt registered [ 4143.670652] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 4145.008143] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4145.229791] alg: No test for adler32 (adler32-zlib) [ 4146.381826] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4146.631632] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 4148.319332] Key type lgssc registered [ 4149.885675] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4158.079502] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4158.110163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4159.607105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4164.962781] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4170.220041] LustreError: 141972:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4170.260281] LustreError: 141972:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4175.345023] LustreError: 141971:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4177.341515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4177.904948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4183.351363] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4191.292978] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4197.169106] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4203.959992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4204.378495] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4205.433390] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4211.689345] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4220.982099] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4222.846805] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4230.119552] Lustre: 144556:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4233.069745] Lustre: 144604:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4234.757912] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4235.752789] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4240.004873] Lustre: 144751:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4240.913186] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4242.241799] Lustre: 144861:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4248.550406] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4248.567712] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.570892] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4248.594443] Lustre: Skipped 1 previous similar message [ 4253.300887] Lustre: server umount lustre-OST0000 complete [ 4258.790937] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4258.796595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4258.807298] Lustre: Skipped 1 previous similar message [ 4258.815604] Lustre: Skipped 1 previous similar message [ 4263.905722] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4263.911568] Lustre: Skipped 2 previous similar messages [ 4269.049344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4269.061436] Lustre: Skipped 1 previous similar message [ 4271.071160] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4271.393533] Lustre: server umount lustre-MDT0000 complete [ 4274.149340] LustreError: 143134:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4274.186735] LustreError: 143134:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4275.771482] LustreError: 144654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448349 with bad export cookie 11625141548340040319 [ 4275.779502] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4275.798579] LustreError: 144654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4276.197317] Lustre: server umount lustre-MDT0001 complete [ 4286.866672] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 4289.950204] Key type lgssc unregistered [ 4290.252898] LNet: 145918:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4290.260929] LNetError: 145918:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4291.308040] LNet: Removed LNI 192.168.204.123@tcp [ 4292.271228] Key type .llcrypt unregistered [ 4292.275238] Key type ._llcrypt unregistered [ 4316.722960] Key type ._llcrypt registered [ 4316.727748] Key type .llcrypt registered [ 4316.865649] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 4318.154888] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4318.170571] alg: No test for adler32 (adler32-zlib) [ 4319.140287] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4319.384832] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 4321.055304] Key type lgssc registered [ 4322.286325] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4330.459909] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4330.496643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4332.230164] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4336.904753] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4341.735405] LustreError: 147336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4341.759320] LustreError: 147336:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4346.077040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4346.601198] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4350.706258] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4358.498732] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4363.685524] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4370.573553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4371.006475] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4372.031393] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4377.145844] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4377.614968] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4385.025916] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4390.813197] Lustre: *** cfs_fail_loc=159, val=0*** [ 4407.107983] Lustre: lustre-MDT0000: Client 3fb67aaf-3ac9-46a7-b95f-11a36410c016 (at 192.168.204.23@tcp) reconnecting [ 4428.260461] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4428.264817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4428.267860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4428.279241] Lustre: Skipped 1 previous similar message [ 4430.247990] Lustre: server umount lustre-OST0000 complete [ 4438.498335] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4438.504151] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4438.504585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4438.504590] Lustre: Skipped 1 previous similar message [ 4438.524050] Lustre: Skipped 1 previous similar message [ 4438.539152] LustreError: Skipped 1 previous similar message [ 4441.085089] Lustre: server umount lustre-MDT0000 complete [ 4443.618699] LustreError: 147336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4443.664268] LustreError: 147336:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4446.168304] LustreError: 147317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448519 with bad export cookie 15649567235138349146 [ 4446.180978] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4446.181579] LustreError: 147317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4446.541856] Lustre: server umount lustre-MDT0001 complete [ 4461.192216] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 4465.168653] Key type lgssc unregistered [ 4465.640748] LNet: 151021:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4465.659505] LNetError: 151021:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4465.686631] LNet: Removed LNI 192.168.204.123@tcp [ 4466.978421] Key type .llcrypt unregistered [ 4466.980200] Key type ._llcrypt unregistered [ 4498.049532] Key type ._llcrypt registered [ 4498.053765] Key type .llcrypt registered [ 4498.253757] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 4499.405430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4499.574350] alg: No test for adler32 (adler32-zlib) [ 4500.813233] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4501.116797] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 4502.851702] Key type lgssc registered [ 4504.282921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4513.511401] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4513.543237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4515.368393] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4521.163868] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4524.513996] LustreError: 152438:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4524.540824] LustreError: 152438:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4529.634020] LustreError: 152439:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4530.273048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4530.790940] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4536.153665] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4544.997670] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4551.175124] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4558.263102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4558.772600] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4559.810067] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4565.816399] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4573.260437] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4576.241188] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4598.069334] Lustre: lustre-OST0000: evicting 515129b8-a68b-4c25-8368-29270211ff75 (at 192.168.204.23@tcp) by administrative request [ 4598.077864] Lustre: Skipped 1 previous similar message [ 4627.427186] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4627.431667] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4627.435802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4627.446945] Lustre: Skipped 1 previous similar message [ 4632.552415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4632.559774] Lustre: Skipped 2 previous similar messages [ 4637.665735] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4637.683195] Lustre: Skipped 1 previous similar message [ 4641.247169] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4641.508323] Lustre: server umount lustre-OST0000 complete [ 4647.907598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4647.929056] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4647.932700] Lustre: Skipped 1 previous similar message [ 4647.956040] Lustre: Skipped 1 previous similar message [ 4653.026242] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4653.034057] Lustre: Skipped 1 previous similar message [ 4660.191534] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4660.388347] Lustre: server umount lustre-MDT0000 complete [ 4663.275695] LustreError: 153168:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4663.291899] LustreError: 153168:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4664.103712] LustreError: 152420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448737 with bad export cookie 8117703594950076576 [ 4664.104728] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4664.113163] LustreError: 152420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4664.516305] Lustre: server umount lustre-MDT0001 complete [ 4678.435557] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 4681.585273] Key type lgssc unregistered [ 4681.952706] LNet: 156514:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4681.965551] LNetError: 156514:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4681.993180] LNet: Removed LNI 192.168.204.123@tcp [ 4683.145411] Key type .llcrypt unregistered [ 4683.149673] Key type ._llcrypt unregistered [ 4759.365503] Key type ._llcrypt registered [ 4759.367720] Key type .llcrypt registered [ 4759.496509] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 4760.611558] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4760.708368] alg: No test for adler32 (adler32-zlib) [ 4761.829862] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4762.125273] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 4763.783192] Key type lgssc registered [ 4764.794811] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4772.120133] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4772.137487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4773.522870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4778.640599] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4784.105562] LustreError: 159003:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4784.139866] LustreError: 159003:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4788.288146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4788.971930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4794.764706] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4802.698764] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4809.742913] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4817.686028] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4818.129468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4819.148864] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4826.951663] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 4833.493242] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4837.978775] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4850.144197] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4850.161040] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4850.186149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4854.248072] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4854.249586] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4854.257146] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4856.385220] Lustre: server umount lustre-OST0000 complete [ 4860.384741] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4860.392261] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4860.405446] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4860.411289] Lustre: Skipped 1 previous similar message [ 4864.480644] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4864.481691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4864.498406] Lustre: Skipped 1 previous similar message [ 4865.998536] Lustre: server umount lustre-MDT0000 complete [ 4869.600824] LustreError: 158998:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4869.627363] LustreError: 158998:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4870.323142] LustreError: 158984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448944 with bad export cookie 3348251929216447770 [ 4870.338959] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4870.880612] Lustre: server umount lustre-MDT0001 complete [ 4885.786864] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 4888.854315] Key type lgssc unregistered [ 4889.223678] LNet: 162551:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4889.237938] LNetError: 162551:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4889.257303] LNet: Removed LNI 192.168.204.123@tcp [ 4890.338607] Key type .llcrypt unregistered [ 4890.343088] Key type ._llcrypt unregistered [ 4917.908230] Key type ._llcrypt registered [ 4917.913257] Key type .llcrypt registered [ 4918.030419] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 4919.497534] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4919.517105] alg: No test for adler32 (adler32-zlib) [ 4920.619904] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4920.862337] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 4922.519184] Key type lgssc registered [ 4923.845193] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4939.350581] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:23:31 (1788449011) [ 4956.930394] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 4968.633285] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 5004.955984] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5004.990643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5006.202130] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5006.262785] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5006.398289] Lustre: lustre-MDT0000: new disk, initializing [ 5006.552576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5006.587403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5011.674259] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 5023.971109] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5024.135455] Lustre: 166812:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5024.237223] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5024.242150] Lustre: Skipped 1 previous similar message [ 5024.353821] Lustre: lustre-MDT0001: new disk, initializing [ 5024.489929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5024.555955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5024.573727] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5030.204419] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 5041.500681] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5048.246785] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5055.862469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5056.151127] Lustre: lustre-OST0000: new disk, initializing [ 5056.154795] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5056.169767] Lustre: 168783:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5056.274799] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5062.262519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5062.275767] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5062.445905] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5063.092406] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 5073.558130] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5083.110769] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5083.111548] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5083.112910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5083.120021] Lustre: Skipped 1 previous similar message [ 5083.140648] LustreError: Skipped 1 previous similar message [ 5088.228944] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5088.245424] Lustre: Skipped 2 previous similar messages [ 5088.598710] Lustre: server umount lustre-OST0000 complete [ 5093.346287] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5093.362732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5093.372049] Lustre: Skipped 1 previous similar message [ 5098.474310] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5098.494870] Lustre: Skipped 2 previous similar messages [ 5100.643323] Lustre: server umount lustre-MDT0000 complete [ 5104.906972] LustreError: 166805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449178 with bad export cookie 11161274889310685541 [ 5104.908771] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5104.914471] LustreError: 166805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5105.435067] Lustre: server umount lustre-MDT0001 complete [ 5118.413758] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 5121.312915] Key type lgssc unregistered [ 5121.543792] LNet: 170691:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5121.553335] LNetError: 170691:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5121.579776] LNet: Removed LNI 192.168.204.123@tcp [ 5122.616969] Key type .llcrypt unregistered [ 5122.619640] Key type ._llcrypt unregistered [ 5147.272543] Key type ._llcrypt registered [ 5147.275358] Key type .llcrypt registered [ 5147.363839] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 5148.704352] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5148.754993] alg: No test for adler32 (adler32-zlib) [ 5149.768626] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 5150.022827] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 5151.703203] Key type lgssc registered [ 5152.829264] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5161.268410] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5161.296281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5162.749583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5167.823571] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 5173.222838] LustreError: 172108:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5173.237964] LustreError: 172108:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5176.006647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5176.393559] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5180.376801] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 5187.754708] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5193.607062] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5200.486455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5201.010283] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5207.673891] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 5208.193642] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5216.661755] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5223.959558] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5223.988453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5224.028368] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5224.047117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.067233] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5225.208909] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5225.233140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5225.271585] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5225.292072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.315304] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5226.456348] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5226.479586] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5226.494231] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5226.509618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.525496] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5227.521360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.452545] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5228.457783] Lustre: Skipped 1 previous similar message [ 5228.478671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.320694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5229.338538] Lustre: Skipped 2 previous similar messages [ 5229.348459] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5229.352854] Lustre: Skipped 2 previous similar messages [ 5229.357995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.389355] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5229.393411] Lustre: Skipped 2 previous similar messages [ 5230.316064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.389868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.386388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.347350] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5233.366932] Lustre: Skipped 4 previous similar messages [ 5233.388385] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5233.394454] Lustre: Skipped 3 previous similar messages [ 5233.397365] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5233.400763] Lustre: Skipped 3 previous similar messages [ 5233.404345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.411666] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5233.419357] Lustre: Skipped 3 previous similar messages [ 5234.455578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.278635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.111706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.015027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.935558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.865903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.613121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.471134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.709496] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5241.716462] Lustre: Skipped 8 previous similar messages [ 5241.747934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5241.761973] Lustre: Skipped 8 previous similar messages [ 5241.769079] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5241.775243] Lustre: Skipped 8 previous similar messages [ 5241.784254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.800300] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5241.813281] Lustre: Skipped 8 previous similar messages [ 5242.909457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.882859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.810126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.797445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.944443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.905528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.756763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.883472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.824866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.912354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.852074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.819635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.622980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.581644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.521553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.527611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.325293] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5258.329428] Lustre: Skipped 16 previous similar messages [ 5258.339539] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5258.348710] Lustre: Skipped 16 previous similar messages [ 5258.354473] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5258.363047] Lustre: Skipped 16 previous similar messages [ 5258.369576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.392304] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5258.401794] Lustre: Skipped 16 previous similar messages [ 5259.290893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.148309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.054715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.058923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.219149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.132582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.035931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.874725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.708049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.674278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.643698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.718100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.537213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.468527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.341926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.294878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.042184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.954503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.913618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.893715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.947097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.089923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.923586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.719673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.709408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.487184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.286528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.278585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.147043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.185288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.072245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.348381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.690788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.852984] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5290.861055] Lustre: Skipped 33 previous similar messages [ 5290.886780] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5290.903230] Lustre: Skipped 33 previous similar messages [ 5290.917602] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5290.928146] Lustre: Skipped 33 previous similar messages [ 5290.934034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.962598] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5290.982686] Lustre: Skipped 33 previous similar messages [ 5292.193426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.422832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.490815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.593262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.728775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.705367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.957487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.474880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.759229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.699200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.668213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.912956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.097812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.983657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.034894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.031739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.202351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.118475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.096247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.084493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.087744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.059988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.082577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.084252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.916256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.785349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.885701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.840764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.830976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.804325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.855453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.712249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.812172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.099513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.091826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.211890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.344804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.667714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.606167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.666089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.718777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.875694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.799275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.837988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.693906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.486966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.310269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.200680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.084617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.972392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.760907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.591145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.482216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.352352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.312843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.123362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.104774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.895357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.694688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.517724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.350813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.066925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.886626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.761580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.667149] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5355.680223] Lustre: Skipped 64 previous similar messages [ 5355.694990] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5355.707179] Lustre: Skipped 64 previous similar messages [ 5355.711358] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5355.715287] Lustre: Skipped 64 previous similar messages [ 5355.718837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.732599] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5355.751947] Lustre: Skipped 64 previous similar messages [ 5356.723565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.570130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.451718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.199877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.860858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.653882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.417718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.380734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.307680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.097310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.959201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.861233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.887192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.764201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.717981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.550419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.461720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.327938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.359296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.291542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.210709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.061398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.969312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.724534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.621838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.390476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.371290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.151860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.072433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.842893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.699394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.672302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.737612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.750743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.626195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.856973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.771313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.700684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.620487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.623545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.513212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.465165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.525838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.634168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.854770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.171177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.173674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.134662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.980805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.231300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.577725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.568719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.456272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.306318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.186537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.080890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.955726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.878905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.804699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.581459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.516118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.260291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.344584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.244333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.307117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.395567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.324583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.547366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.478242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.374603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.196729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.155564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.027525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.957440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.133294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.007549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.959773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.828639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.776586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.688198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.784563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.600356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.616340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.641461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.711262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.917541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.768381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.912671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.977903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.937065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.311804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.344114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.239457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.473036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.449801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.395250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.381439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.528746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.438105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.544357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.362314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.276748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.171130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.275741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.258584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.302487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.277582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.306336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.193333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.996934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.796443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.668333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.485827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.424433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.324221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.209880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.161718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.103474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.036266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.066961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.023830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.258891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.183607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.232885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.157947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.300967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.092333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.058644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.834313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.895480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.584195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.585340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.483225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.415789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.410802] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5484.419738] Lustre: Skipped 134 previous similar messages [ 5484.453829] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5484.475982] Lustre: Skipped 134 previous similar messages [ 5484.484449] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5484.488982] Lustre: Skipped 134 previous similar messages [ 5484.500136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.523036] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5484.534903] Lustre: Skipped 134 previous similar messages [ 5485.594554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.631258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.736458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.643500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.841663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.950502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.900340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.886616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.851800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.708054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.666932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.529027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.350995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.126760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.145818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.963200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.925738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.722688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.606657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.849901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.900130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.048581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.961418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.907358] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5507.914835] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5507.918060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.805504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.985597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.951689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.798471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.700626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.494859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.492741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.553479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.263273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.041061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.001518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.909935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.877580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.736335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.586965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.445539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.225168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.026917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.947341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.761667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.582259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.284043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.046543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.893271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.812571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.710343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.756939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.599772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.661100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.727719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.866774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.919735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.964588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.850653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.851417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.719866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.471660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.142945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.966048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.770103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.667104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.454128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.333635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.113894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.030809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.944710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.976382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.811245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.645196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.469035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.425970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.335410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.242708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.173556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.045806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.972839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.998045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.981492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.035883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.080192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.934861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.878787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.749145] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5564.762272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.763337] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5565.664215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.654487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.546680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.577852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.422905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.582110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.755112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.821041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.852028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.062559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.919114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.150826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.335631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.437227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.627431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.341467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.327879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.377609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.312646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.381887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.433500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.879498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.077105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.000492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.023133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.824238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.619880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.870324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.441065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.027617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.524579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.048466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.495568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.841862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.291617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.566164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.936490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.550448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.875937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.741165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.794772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.704540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.978742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.955703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.395329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.593141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.648802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.700654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.940247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.275486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.319326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.515488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.786501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.982165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.757085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.888028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.832516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.818183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.617335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.631454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.332433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.223574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.163773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.393448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.506918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.798053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.836870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.432136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.257809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.091474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.977591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.882897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.793286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.542639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.356695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.303595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.162962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.037541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.918432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.952415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.793151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.852912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.644776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.599280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.360883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.180327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.091716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.074838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.903691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.898424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.864560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.642191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.436022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.233317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.165671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.998282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.690044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.433335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.163222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.881684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.621645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.445483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.120532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.017595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.819701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.621914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.325187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.077172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.898624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.692935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.685596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.536712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.291362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.106545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.867816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.706656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.479821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.394224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.162047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.926853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.756860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.378857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.131443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.004752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.005443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.907968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.870333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.731694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.577285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.285024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.111576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.092426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.978466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.807994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.828621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.940323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.012251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.083960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.888500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.656525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.349923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.257558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.033333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.005105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.103427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.964289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.100650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.127317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.973176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.858225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.666105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.806692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.678322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.774271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.928894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.215762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.161611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.381108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.667767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.050624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.070667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.129936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.349820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.622588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.545982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.832381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.001214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.960091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.380655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.698881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.688977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.756128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.691514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.759856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.713604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.959959] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5740.966473] Lustre: Skipped 262 previous similar messages [ 5740.989850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5741.012348] Lustre: Skipped 262 previous similar messages [ 5741.017456] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5741.025196] Lustre: Skipped 262 previous similar messages [ 5741.031525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.042948] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5741.045019] Lustre: Skipped 262 previous similar messages [ 5742.041787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.065810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.958955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.094428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.222832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.552580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.861403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.897243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.889531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.862335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.893215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.057852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.118961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.418595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.386232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.488423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.436304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.377462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.368589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.198112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.028237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.938626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.104913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.092681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.957473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.759796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.664274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.551867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.367176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.170511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.015822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.897155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.678798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.404299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.134146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.910553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.701321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.475423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.289649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.057307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.788492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.536315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.309427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.111299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.895854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.693697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.521937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.279617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.135224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.942238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.814691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.580015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.558343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.372249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.216816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.991040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.811558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.648308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.586758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.313404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.065654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.924657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.902131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.621696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.439788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.269818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.164786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.035066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.951551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.673461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.606701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.482434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.450524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.229302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.101647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.011801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.898396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.049763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.933178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.830299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.751725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.739300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.585248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.406811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.231676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.048771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.795609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.572438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.314972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.278253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.224231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.010517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.746567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.516481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.286535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.166126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.999547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.871085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.939388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.884087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.918552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.730817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.560647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.355409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.334712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.233858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.045836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.044909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.034420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.948921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.764389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.548771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.456311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.428387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.269826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.121787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.846240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.607518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.296730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.050386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.826654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.599712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.362938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.234874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.087208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.953509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.671075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.565833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.357941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.226375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.095706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.004247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.916244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.882085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.645260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.568285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.475535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.503440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.348453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.175201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.014752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.846838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.543655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.401441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.268592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.280951] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5869.299384] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5869.304938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.092490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.967243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.750945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.675282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.505858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.325648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.095134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.950680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.787221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.588051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.439484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.392678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.235367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.216250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.262475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.177778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.996905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.893617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.624380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.600682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.563975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.662822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.600555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.633907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.428353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.427890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.275201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.055598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.975897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.090884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.888859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.797587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.515616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.202089] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5899.207873] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5899.220665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.173175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.049449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.772489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.789469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.625925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.507306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.401577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.312407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.099672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.924831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.738902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.543950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.411970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.260597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.078153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.980516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.756731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.657989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.415671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.150806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.930708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.828586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.602948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.484941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.277871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.082952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.828409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.690267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.568511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.486729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.472237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.312351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.097215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.027265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.810238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.670450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.542705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.386537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.210933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.974244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.709739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.724826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.673582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.555217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.386363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.237392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.223263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.304041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.166665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.099440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.126977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.114437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.079864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.160761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.982223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.026586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.995046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.059724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.089856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.954963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.904058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.813987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.782157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.663517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.666158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.612489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.525212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.441990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.280514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.170585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.056390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.004694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.866904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.712871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.667309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.675458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.662349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.445702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.287075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.141051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.189861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.376333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.473768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.452202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.445748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.526129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.384566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.453547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.427475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.252662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.183784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.086138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.073583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.032747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.908964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.748314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.650658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.876845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.969286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.920698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.822938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.760094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.855918] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5992.870849] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5992.893631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.876852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.789302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.644720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.583888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.364259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.267496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.086332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.895382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.725415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.645539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.492326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.425994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.559500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.357507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.333514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.107781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.965379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.783782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.563468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.356596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.170504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.047853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.823819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.605986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.560804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.456280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.325716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.146552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.876707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.714531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.677444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.546900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.505441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.282253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.329727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.253611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.322430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.170031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.221928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.180411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.175794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.096816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.142726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.200709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.268313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.253585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.205544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.061397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.050251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.020941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.977716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.903424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.917068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.855268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.985984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.923229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.001446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.959559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.817862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.827574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.703785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.652938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.586976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.506055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.409725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.484815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.545637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.541878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.475425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.434277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.503386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.586552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.566876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.412902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.386515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.407671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.502858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.563446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.579537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.719588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.693404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.889748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.923449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.926035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.225588] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6073.237745] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6073.255866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.478601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.763505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.070342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.247268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.457221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.557334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.855843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.877225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.961702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.188844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.307400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.489260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.422364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.531502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.391370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.410464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.590302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.536461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.470696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.385796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.318345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.213691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.302965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.076197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.317831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.297186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.328190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.317907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.320412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.304879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.290972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.283888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.294403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.381961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.446824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.441937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.515310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.462345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.436326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.479208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.702871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.749967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.762213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.777477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.810678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.751436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.706583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.647837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.594331] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6124.608364] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6124.608738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.590504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.647562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.609800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.797129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.695317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.790538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.742450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.744511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.869325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.751578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.771080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.687641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.691528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.623755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.886951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.035556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.900747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.123680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.142552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.116481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.219321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.163267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.162604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.149568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.387583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.451045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.635476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.874956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.119404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.463711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.825732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.963235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.092518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.192436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.347796] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6162.364663] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6162.372481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.538865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.580381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.732756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.875498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.754488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.757792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.866241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.081356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.979538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.911908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.832765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.944035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.834932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.757751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.708673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.404218] LustreError: 173930:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6556.419896] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7220.707231] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7220.711240] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7220.721819] Lustre: Skipped 467 previous similar messages [ 7220.740624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7222.759769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7226.674900] Lustre: server umount lustre-OST0000 complete [ 7232.999033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7233.011242] Lustre: Skipped 2 previous similar messages [ 7237.097319] Lustre: server umount lustre-MDT0000 complete [ 7238.122685] LustreError: 172109:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7238.148265] LustreError: 172109:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7240.803890] LustreError: 172088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451314 with bad export cookie 7838725312386903603 [ 7240.807841] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7240.810235] LustreError: 172088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7241.193632] Lustre: server umount lustre-MDT0001 complete [ 7253.365113] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 7256.815748] Key type lgssc unregistered [ 7257.142895] LNet: 227380:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7257.160957] LNetError: 227380:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7258.235396] LNet: Removed LNI 192.168.204.123@tcp [ 7259.557098] Key type .llcrypt unregistered [ 7259.558691] Key type ._llcrypt unregistered [ 7285.833457] Key type ._llcrypt registered [ 7285.836919] Key type .llcrypt registered [ 7286.000717] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 7287.304289] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7287.536239] alg: No test for adler32 (adler32-zlib) [ 7288.619626] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7288.880494] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 7290.551219] Key type lgssc registered [ 7292.005921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7300.751373] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7300.777871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7302.262943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7306.854815] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7311.845191] LustreError: 228799:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7311.866659] LustreError: 228799:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7316.223579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7316.605644] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7321.043954] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7328.459629] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7335.684801] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7344.310319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7344.855372] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7346.952426] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7352.688734] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7362.079708] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7367.878806] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7370.105626] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7370.118272] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7370.131106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.147631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7370.148548] LustreError: 228818:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7374.636367] Lustre: lustre-OST0000: Not available for connect from 192.168.204.23@tcp (stopping) [ 7376.865343] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7376.875399] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7376.878628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7376.896786] Lustre: Skipped 1 previous similar message [ 7378.483546] Lustre: server umount lustre-OST0000 complete [ 7390.223137] LustreError: 228818:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7413.079188] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7413.500753] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7416.566594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.576525] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.590509] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7416.595243] Lustre: Skipped 1 previous similar message [ 7420.217336] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7427.715396] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7429.295771] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7436.320569] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0c885dc4000.ost_server_uuid 50 [ 7437.797741] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0c885dc4000.ost_server_uuid in IDLE state after 0 sec [ 7442.403082] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7442.416084] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7442.423069] Lustre: Skipped 1 previous similar message [ 7442.434643] Lustre: Skipped 1 previous similar message [ 7447.522415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7447.534232] Lustre: Skipped 1 previous similar message [ 7448.319588] Lustre: server umount lustre-OST0000 complete [ 7454.694434] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7454.708137] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7454.723259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7459.055943] Lustre: server umount lustre-MDT0000 complete [ 7462.882764] LustreError: 228795:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7462.901766] LustreError: 228795:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7463.603734] LustreError: 229535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451537 with bad export cookie 8760052160119554555 [ 7463.610313] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7463.617634] LustreError: 229535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7464.106482] Lustre: server umount lustre-MDT0001 complete [ 7477.042239] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 7480.173562] Key type lgssc unregistered [ 7480.517793] LNet: 234013:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7480.525609] LNetError: 234013:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7480.552258] LNet: Removed LNI 192.168.204.123@tcp [ 7481.704339] Key type .llcrypt unregistered [ 7481.709731] Key type ._llcrypt unregistered [ 7526.257462] Key type ._llcrypt registered [ 7526.262211] Key type .llcrypt registered [ 7526.386672] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 7527.543632] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7527.600749] alg: No test for adler32 (adler32-zlib) [ 7528.814105] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7529.080567] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 7530.735362] Key type lgssc registered [ 7531.907196] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7541.784195] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7541.822622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7543.335739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7548.529988] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7553.505701] LustreError: 235744:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7553.543377] LustreError: 235744:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7558.022562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7558.607991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7564.466553] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7567.700862] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7574.670830] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7575.131809] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7582.180137] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7588.480498] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7592.897078] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7593.085983] Lustre: lustre-OST0001: new disk, initializing [ 7593.088729] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7593.093410] Lustre: 238051:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7593.200494] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7600.350213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7600.364920] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7600.488757] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7601.598502] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7614.387629] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7624.115455] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7640.549127] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7640.560606] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7640.581713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7641.569594] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7641.577058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7641.586603] Lustre: Skipped 1 previous similar message [ 7641.604043] Lustre: Skipped 3 previous similar messages [ 7645.435531] Lustre: server umount lustre-MDT0000 complete [ 7646.688591] LustreError: 235744:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7649.434658] LustreError: 238823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451723 with bad export cookie 16806008630786672538 [ 7649.436480] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7649.450487] LustreError: 238823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7649.787987] Lustre: server umount lustre-MDT0001 complete [ 7663.843906] Lustre: server umount lustre-OST0000 complete [ 7678.790405] Lustre: server umount lustre-OST0001 complete [ 7706.468816] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 7718.600231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7719.121565] LustreError: 241169:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7719.141440] LustreError: 241169:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7719.239896] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7724.231416] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7724.527356] LustreError: 241170:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7728.616574] LustreError: 241169:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7734.655360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7735.178660] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7739.790265] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7742.882338] Lustre: 242313:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7750.214406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7757.549541] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7759.793957] LustreError: 242667:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7759.807993] LustreError: 242667:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7759.820309] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7763.912642] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7767.873691] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7768.219632] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7768.227773] Lustre: Skipped 1 previous similar message [ 7773.703032] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7775.218801] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7783.407363] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7787.291933] Lustre: 244182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7799.280110] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7799.287923] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7799.290904] Lustre: Skipped 2 previous similar messages [ 7803.960040] Lustre: server umount mylustre-MDT0000 complete [ 7804.406668] LustreError: 241165:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7804.434445] LustreError: 241165:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 7807.962496] LustreError: 241145:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451881 with bad export cookie 16806008630786675401 [ 7807.966234] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7807.977045] LustreError: 241145:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7808.307669] Lustre: server umount mylustre-MDT0001 complete [ 7822.911319] Lustre: server umount mylustre-OST0000 complete [ 7825.889478] Lustre: 235006:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788451883/real 1788451883] req@ffff8ac6b77ac700 x1875327427182848/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788451899 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7825.944620] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7825.961841] Lustre: Skipped 2 previous similar messages [ 7827.658884] Lustre: server umount mylustre-OST0001 complete [ 7850.027243] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 7861.228502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7861.973963] LustreError: 246491:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7862.175576] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7869.093321] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7880.802143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7886.129854] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7889.420139] Lustre: 247633:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7896.686192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7905.049698] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7905.345253] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7907.401715] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7916.280907] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7922.236902] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7924.384249] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 7934.395896] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7939.213859] Lustre: 249504:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7947.750824] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7947.754092] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7947.758411] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7947.758420] Lustre: Skipped 3 previous similar messages [ 7947.762208] Lustre: Skipped 2 previous similar messages [ 7952.871862] LustreError: 246488:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7952.908449] LustreError: 246488:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 7952.928740] Lustre: server umount tfs-MDT0000 complete [ 7956.110130] LustreError: 246466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452029 with bad export cookie 16806008630786677578 [ 7956.115562] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7956.120663] LustreError: 246466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7956.369235] Lustre: server umount tfs-MDT0001 complete [ 7970.284819] Lustre: server umount tfs-OST0000 complete [ 7973.355196] Lustre: 235005:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788452031/real 1788452031] req@ffff8ac583136680 x1875327427238144/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788452047 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7973.387746] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7973.401552] Lustre: Skipped 1 previous similar message [ 7974.352313] Lustre: server umount tfs-OST0001 complete [ 7997.460855] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 8007.696390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8008.288406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8008.300325] Lustre: Skipped 3 previous similar messages [ 8014.470231] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8025.032568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8030.501279] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8033.561504] Lustre: 252902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8040.513487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8047.875606] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8051.195687] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8053.245181] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8056.678702] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8062.468755] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8063.381898] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8071.277112] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8074.879443] Lustre: 254772:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8083.584134] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:15:54 (1788452154) [ 8107.492072] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8107.514047] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8107.561307] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8107.575272] Lustre: Skipped 3 previous similar messages [ 8118.241010] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8118.548168] Lustre: server umount lustre-MDT0000 complete [ 8118.759683] LustreError: 251753:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8118.787823] LustreError: 251753:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 8123.165737] LustreError: 254083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452196 with bad export cookie 16806008630786679496 [ 8123.176704] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8123.181783] LustreError: 254083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8123.879442] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8123.885699] Lustre: Skipped 9 previous similar messages [ 8129.592710] Lustre: server umount lustre-MDT0001 complete [ 8140.529678] Lustre: server umount lustre-OST0000 complete [ 8150.368983] Lustre: server umount lustre-OST0001 complete [ 8156.934716] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 8167.538618] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 8204.327320] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8204.327320] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8204.327320] [ 8204.360367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8204.630428] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8204.690616] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8204.791487] Lustre: lustre-MDT0000: new disk, initializing [ 8204.924064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8204.933793] Lustre: Skipped 3 previous similar messages [ 8204.962786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8209.284839] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8219.733256] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8219.733256] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8219.733256] [ 8219.758092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8219.898262] Lustre: 257955:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8219.958810] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8219.962221] Lustre: Skipped 1 previous similar message [ 8220.064039] Lustre: lustre-MDT0001: new disk, initializing [ 8220.251616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8220.264249] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8224.989209] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8238.966402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8239.580392] Lustre: lustre-OST0000: new disk, initializing [ 8239.585475] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8239.609345] Lustre: 259536:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8241.192352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8241.218279] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8241.477021] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8248.316227] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8258.979057] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8259.229896] Lustre: lustre-OST0001: new disk, initializing [ 8259.239934] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8259.248754] Lustre: 260409:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8260.798488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8260.820134] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8260.973717] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8266.445447] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8278.776457] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:19:10 (1788452350) [ 8295.526862] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:19:27 (1788452367) [ 8312.301077] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8312.309861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8312.312726] Lustre: Skipped 6 previous similar messages [ 8312.322421] Lustre: Skipped 3 previous similar messages [ 8317.723094] Lustre: server umount lustre-OST0000 complete [ 8322.550303] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8328.527058] Lustre: server umount lustre-MDT0000 complete [ 8332.793609] LustreError: 257948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452406 with bad export cookie 16806008630786681274 [ 8332.802607] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8332.808231] LustreError: 257948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8339.215276] Lustre: server umount lustre-MDT0001 complete [ 8350.562891] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 8423.949667] Lustre: server umount lustre-OST0001 complete [ 8431.556311] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 8446.824215] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 8481.297905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8481.563265] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8481.606667] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8481.675896] Lustre: lustre-MDT0000: new disk, initializing [ 8481.758446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8481.764997] Lustre: Skipped 3 previous similar messages [ 8481.786978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8486.215586] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8497.908986] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8498.022674] Lustre: 265206:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8498.105151] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8498.111598] Lustre: Skipped 1 previous similar message [ 8498.204750] Lustre: lustre-MDT0001: new disk, initializing [ 8498.378471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8498.401393] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8504.100361] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8515.081997] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8521.588822] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8529.571333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8529.827382] Lustre: lustre-OST0000: new disk, initializing [ 8529.830576] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8529.835407] Lustre: 267177:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8531.336515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8531.372562] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8531.465299] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8537.333803] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8548.923118] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8557.033807] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8557.037665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8557.043770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8557.043779] Lustre: Skipped 9 previous similar messages [ 8557.045680] LustreError: Skipped 1 previous similar message [ 8557.099805] Lustre: Skipped 5 previous similar messages [ 8562.758453] Lustre: server umount lustre-OST0000 complete [ 8573.682905] Lustre: server umount lustre-MDT0000 complete [ 8577.508084] LustreError: 265217:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8577.557773] LustreError: 265217:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 8577.908249] LustreError: 265198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452651 with bad export cookie 16806008630786682744 [ 8577.914704] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8577.946893] LustreError: 265198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8578.496173] Lustre: server umount lustre-MDT0001 complete [ 8594.924277] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 8599.043396] Key type lgssc unregistered [ 8599.512658] LNet: 268942:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8599.529910] LNetError: 268942:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8600.551864] LNet: Removed LNI 192.168.204.123@tcp [ 8601.793171] Key type .llcrypt unregistered [ 8601.797094] Key type ._llcrypt unregistered [ 8633.856459] Key type ._llcrypt registered [ 8633.858463] Key type .llcrypt registered [ 8633.964943] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 8635.499415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8635.552114] alg: No test for adler32 (adler32-zlib) [ 8636.612876] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8636.952587] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 8638.623185] Key type lgssc registered [ 8639.997313] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8648.861797] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8648.884596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8650.359422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8655.626163] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8660.966381] LustreError: 270408:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8660.995184] LustreError: 270408:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8664.708750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8665.103362] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8670.243598] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8678.312924] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8684.457102] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8691.593065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8691.991265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8696.117472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8698.646343] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8707.094330] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8708.266576] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8711.333223] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8711.337920] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8716.568174] Lustre: server umount lustre-OST0000 complete [ 8721.380868] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8721.392196] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8721.416732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8721.421717] Lustre: Skipped 1 previous similar message [ 8727.012669] LustreError: 270407:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8727.097342] Lustre: server umount lustre-MDT0000 complete [ 8731.148909] LustreError: 270388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452804 with bad export cookie 11021578368352274697 [ 8731.156766] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8731.164803] LustreError: 270388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8731.563938] Lustre: server umount lustre-MDT0001 complete [ 8744.533620] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 8748.162877] Key type lgssc unregistered [ 8748.459228] LNet: 274002:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8748.469420] LNetError: 274002:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8748.503482] LNet: Removed LNI 192.168.204.123@tcp [ 8749.714366] Key type .llcrypt unregistered [ 8749.716052] Key type ._llcrypt unregistered [ 8770.757281] Key type ._llcrypt registered [ 8770.759720] Key type .llcrypt registered [ 8770.936530] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 8772.708040] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8772.763252] alg: No test for adler32 (adler32-zlib) [ 8773.942988] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8774.289891] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 8776.032834] Key type lgssc registered [ 8777.684231] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8788.009734] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8788.040107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8803.552206] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8808.675197] LustreError: 275197:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.123@tcp: failed processing log, type 4: rc = -110 [ 8835.359211] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8842.842887] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 8850.630466] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8852.930516] Lustre: server umount lustre-OST0000 complete [ 8869.372793] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 8872.277605] Key type lgssc unregistered [ 8872.550863] LNet: 276831:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8872.560475] LNetError: 276831:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8872.587663] LNet: Removed LNI 192.168.204.123@tcp [ 8873.473811] Key type .llcrypt unregistered [ 8873.479930] Key type ._llcrypt unregistered [ 8912.108176] Key type ._llcrypt registered [ 8912.117188] Key type .llcrypt registered [ 8912.241212] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 8913.259568] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8913.292587] alg: No test for adler32 (adler32-zlib) [ 8914.352735] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8914.709493] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 8916.375203] Key type lgssc registered [ 8917.581675] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8930.012818] loop0: detected capacity change from 0 to 204800000 [ 8930.309608] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8930.321905] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8930.663227] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8931.678913] loop0: detected capacity change from 0 to 204800000 [ 8931.870564] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8931.879922] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8932.215987] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8933.082134] loop0: detected capacity change from 0 to 204800000 [ 8933.263789] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8933.272921] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8933.382926] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8934.399761] loop0: detected capacity change from 0 to 204800000 [ 8934.621310] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8935.743748] loop0: detected capacity change from 0 to 204800000 [ 8935.773545] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8939.639372] loop0: detected capacity change from 0 to 204800000 [ 8939.679697] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8943.024878] loop0: detected capacity change from 0 to 204800000 [ 8943.069243] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8946.529754] loop0: detected capacity change from 0 to 204800000 [ 8946.560046] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8950.092369] loop0: detected capacity change from 0 to 204800000 [ 8950.164227] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8950.193304] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8950.285698] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8950.298586] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8950.314335] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8950.334646] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8950.343428] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8950.353492] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8950.361763] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8950.372450] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8950.381438] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8950.390729] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8950.400458] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8950.411974] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8950.421878] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8950.436306] Lustre: 280242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8955.808759] Lustre: server umount MGS complete [ 8956.690148] loop0: detected capacity change from 0 to 204800000 [ 8956.795243] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8957.234228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8957.291772] Lustre: lustre-MDT0000: Aborting client recovery [ 8958.105738] loop1: detected capacity change from 0 to 204800000 [ 8958.257528] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8958.344690] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8958.360678] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8958.378992] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8958.392751] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8958.403555] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8958.412506] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8958.420726] Lustre: 280675:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8958.436299] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8958.573454] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8958.617657] Lustre: lustre-MDT0001: Aborting client recovery [ 8959.481576] loop2: detected capacity change from 0 to 204800000 [ 8959.632913] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8959.710233] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8959.718816] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8959.725129] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8959.741372] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8959.750874] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8959.766122] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8959.776358] Lustre: 280753:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8959.889397] Lustre: Found index 0 for lustre-OST0000, updating log [ 8959.914299] Lustre: 280773:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8959.980526] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8959.997704] Lustre: lustre-OST0000: Aborting recovery [ 8960.832833] loop3: detected capacity change from 0 to 204800000 [ 8961.005320] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8961.053974] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8961.061584] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8961.075406] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8961.093362] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8961.105400] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8961.116445] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8961.123602] Lustre: 280839:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8961.144339] Lustre: Found index 1 for lustre-OST0001, updating log [ 8961.165372] Lustre: 280846:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8961.223760] Lustre: lustre-OST0001: Aborting recovery [ 8966.311336] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8966.331899] LustreError: 280618:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8966.341268] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8966.343676] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8966.380360] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8968.383142] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8968.387854] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8968.880984] LustreError: 280585:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 593us: rc = -2 [ 8969.388151] LustreError: 280586:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 567us: rc = -2 [ 8969.414807] LustreError: 280586:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 8984.047335] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8984.048607] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8984.051180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8984.065851] Lustre: Skipped 3 previous similar messages [ 8988.674574] Lustre: server umount lustre-MDT0000 complete [ 8989.153263] LustreError: 280583:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8989.181033] LustreError: 280583:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 8990.022050] LustreError: 280568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453063 with bad export cookie 15360538873807834531 [ 8990.033041] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8990.058889] LustreError: 280568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8994.274901] LustreError: 280584:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8994.277844] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8994.317048] LustreError: 280584:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8994.369591] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8994.377165] Lustre: Skipped 4 previous similar messages [ 8995.298904] LustreError: 280582:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8995.335314] LustreError: 280582:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8996.806469] Lustre: server umount lustre-MDT0001 complete [ 9005.415984] Lustre: server umount lustre-OST0000 complete [ 9013.005920] Lustre: server umount lustre-OST0001 complete [ 9022.882687] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 12:31:33 (1788453093) [ 9055.330631] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 9065.146530] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 9106.273412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9106.566412] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9106.601658] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9106.728465] Lustre: lustre-MDT0000: new disk, initializing [ 9106.850965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9106.857307] Lustre: Skipped 1 previous similar message [ 9106.897177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9113.076647] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9126.473924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9126.574991] Lustre: 284913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9126.624418] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9126.628169] Lustre: Skipped 1 previous similar message [ 9126.684237] Lustre: lustre-MDT0001: new disk, initializing [ 9126.787112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9126.851369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9126.863204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9131.823608] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9141.529195] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9148.041918] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9156.729359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9157.026981] Lustre: lustre-OST0000: new disk, initializing [ 9157.030765] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9157.036310] Lustre: 286883:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9157.148716] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9164.471255] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9164.492130] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9164.705121] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9164.903769] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9176.255503] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9183.127741] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9193.994879] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9198.527119] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9205.701760] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9251.810157] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9251.827697] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9251.850140] Lustre: Skipped 1 previous similar message [ 9251.859077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9251.876036] Lustre: Skipped 2 previous similar messages [ 9255.560265] Lustre: server umount lustre-OST0000 complete [ 9260.000630] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9260.010630] LustreError: Skipped 1 previous similar message [ 9260.023075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9260.033339] Lustre: Skipped 1 previous similar message [ 9260.040181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9260.045295] Lustre: Skipped 1 previous similar message [ 9266.202604] Lustre: server umount lustre-MDT0000 complete [ 9267.169113] LustreError: 284919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9267.188914] LustreError: 284919:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9270.221390] LustreError: 287739:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453343 with bad export cookie 15360538873807837233 [ 9270.228082] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9270.233676] LustreError: 287739:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9270.636713] Lustre: server umount lustre-MDT0001 complete [ 9279.933716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9284.860657] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9290.544725] Lustre: server umount MGS complete [ 9304.655356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9305.462156] LustreError: 290657:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9305.490210] LustreError: 290670:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9305.562397] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9312.038937] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9315.813646] LustreError: 290658:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9315.882451] LustreError: 290658:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9322.336249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9328.224915] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9336.520180] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9342.780071] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9349.641556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9350.178275] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9350.188997] Lustre: Skipped 1 previous similar message [ 9356.918782] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9365.423277] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9397.222160] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9397.237092] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9397.268632] Lustre: Skipped 1 previous similar message [ 9397.279038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9397.290945] Lustre: Skipped 2 previous similar messages [ 9408.483738] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9408.501616] Lustre: Skipped 4 previous similar messages [ 9411.039585] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9411.321235] Lustre: server umount lustre-OST0000 complete [ 9418.735969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9418.759053] Lustre: Skipped 2 previous similar messages [ 9422.015797] Lustre: server umount lustre-MDT0000 complete [ 9423.840278] LustreError: 290658:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9423.861644] LustreError: 290658:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9426.208940] LustreError: 290640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453499 with bad export cookie 15360538873807839186 [ 9426.225847] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9426.229013] LustreError: 290640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9426.999667] Lustre: server umount lustre-MDT0001 complete [ 9440.136202] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 9444.299199] Key type lgssc unregistered [ 9444.644236] LNet: 294608:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9444.650103] LNetError: 294608:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9444.671693] LNet: Removed LNI 192.168.204.123@tcp [ 9445.779896] Key type .llcrypt unregistered [ 9445.784939] Key type ._llcrypt unregistered [ 9487.248597] Key type ._llcrypt registered [ 9487.253232] Key type .llcrypt registered [ 9487.379726] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 9503.025962] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 9504.064599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9504.086487] alg: No test for adler32 (adler32-zlib) [ 9505.351638] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9505.616034] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 9507.360190] Key type lgssc registered [ 9508.794258] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9558.672192] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9558.733401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9560.189348] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9560.232209] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9560.331229] Lustre: lustre-MDT0000: new disk, initializing [ 9560.507084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9560.554219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9566.200461] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9578.692236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9578.907618] Lustre: 298586:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9578.999995] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9579.006443] Lustre: Skipped 1 previous similar message [ 9579.107505] Lustre: lustre-MDT0001: new disk, initializing [ 9579.223989] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9579.326323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9579.343464] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9584.511986] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9595.550292] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9601.961739] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9608.718572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9609.080649] Lustre: lustre-OST0000: new disk, initializing [ 9609.085422] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9609.092093] Lustre: 300553:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9609.201867] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9615.776843] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9617.988736] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9618.002361] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9618.127803] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9627.126849] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9634.716592] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9646.471290] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9650.509753] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9659.411621] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9700.320926] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9700.327820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9700.335735] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9700.344652] Lustre: Skipped 1 previous similar message [ 9703.521817] Lustre: server umount lustre-OST0000 complete [ 9707.488929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9707.501421] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9707.515369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9707.525469] Lustre: Skipped 1 previous similar message [ 9710.560763] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9710.561341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9710.586381] Lustre: Skipped 1 previous similar message [ 9713.399670] Lustre: server umount lustre-MDT0000 complete [ 9717.451716] LustreError: 298577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453791 with bad export cookie 9274433036304371079 [ 9717.456517] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9717.471318] LustreError: 298577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9717.961032] Lustre: server umount lustre-MDT0001 complete [ 9726.176358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9730.350153] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9734.931509] Lustre: server umount MGS complete [ 9745.683219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9746.202552] LustreError: 304378:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9746.219698] LustreError: 304378:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9746.237279] LustreError: 304391:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9746.322936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9750.722803] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9751.525903] LustreError: 304379:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9755.624833] LustreError: 304378:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9759.293758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9759.701301] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9763.997000] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9771.742588] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9777.772405] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9784.525223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9784.848269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9790.854319] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9798.462583] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9826.789536] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9826.798965] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9826.804265] LustreError: Skipped 1 previous similar message [ 9826.820692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9826.824026] Lustre: Skipped 1 previous similar message [ 9826.854543] Lustre: Skipped 1 previous similar message [ 9828.094646] Lustre: server umount lustre-OST0000 complete [ 9836.512778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9836.525314] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9836.545265] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9839.697472] Lustre: server umount lustre-MDT0000 complete [ 9842.145123] LustreError: 306219:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9842.170555] LustreError: 306219:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9843.587287] LustreError: 304358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453917 with bad export cookie 9274433036304373032 [ 9843.591123] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9843.596539] LustreError: 304358:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9843.982559] Lustre: server umount lustre-MDT0001 complete [ 9855.843854] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [ 9858.986496] Key type lgssc unregistered [ 9859.233782] LNet: 308327:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9859.246389] LNetError: 308327:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9859.280874] LNet: Removed LNI 192.168.204.123@tcp [ 9860.106061] Key type .llcrypt unregistered [ 9860.108122] Key type ._llcrypt unregistered [ 9882.658710] Key type ._llcrypt registered [ 9882.661286] Key type .llcrypt registered [ 9882.802954] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [ 9897.516384] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 9899.052300] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9899.158527] alg: No test for adler32 (adler32-zlib) [ 9900.447620] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9900.773670] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [ 9902.455184] Key type lgssc registered [ 9903.897611] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9936.283968] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [ 9953.162745] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9953.238286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9954.560713] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9954.600711] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9954.721662] Lustre: lustre-MDT0000: new disk, initializing [ 9954.813289] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9954.837514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9959.607335] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [ 9971.996599] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9972.324411] Lustre: lustre-OST0000: new disk, initializing [ 9972.330438] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9972.338243] Lustre: Skipped 1 previous similar message [ 9972.354059] Lustre: 312378:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9972.487891] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9978.997684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9979.014254] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9979.058320] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9981.906978] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [10859.490294] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10859.509465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10861.241969] Lustre: server umount lustre-MDT0000 complete [10866.467247] LustreError: 311563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788454940 with bad export cookie 8801093179919208358 [10866.478459] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10866.479545] LustreError: 311563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10866.696661] Lustre: server umount lustre-OST0000 complete [10894.131465] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [10905.254840] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [10944.947385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10945.374437] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10945.417616] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10945.520299] Lustre: lustre-MDT0000: new disk, initializing [10945.713432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10945.768897] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10950.878477] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [10961.868929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10962.079591] Lustre: 316732:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10962.160094] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10962.166765] Lustre: Skipped 1 previous similar message [10962.261557] Lustre: lustre-MDT0001: new disk, initializing [10962.374350] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10962.416539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10962.432155] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10967.048915] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [10976.418531] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10981.695256] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10989.696762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10990.108509] Lustre: lustre-OST0000: new disk, initializing [10990.122624] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10990.136971] Lustre: 318703:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10990.297050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10997.362930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10997.384461] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10997.626921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10998.204609] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11012.786781] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11018.745935] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11019.052403] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11024.877944] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11025.085817] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11028.457327] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11028.460820] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11028.462742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11028.491728] Lustre: Skipped 1 previous similar message [11028.504480] Lustre: Skipped 1 previous similar message [11033.573259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11033.585559] Lustre: Skipped 1 previous similar message [11038.690127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11038.695559] Lustre: Skipped 1 previous similar message [11041.247372] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11041.414931] Lustre: server umount lustre-OST0000 complete [11048.932462] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11048.935543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11048.945732] Lustre: Skipped 1 previous similar message [11051.417702] Lustre: server umount lustre-MDT0000 complete [11054.982976] LustreError: 318702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788455128 with bad export cookie 8801093179922149177 [11054.988943] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11054.993020] LustreError: 318702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11055.315916] Lustre: server umount lustre-MDT0001 complete [11058.732307] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11060.487315] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 13:05:32 (1788455132) [11067.482994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11068.066309] LustreError: 321018:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11068.093870] LustreError: 321018:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11068.128809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11072.785513] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11073.509670] LustreError: 321019:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11077.621517] LustreError: 321018:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11081.092242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11081.470849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11085.960788] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11093.491396] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11099.735880] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11107.105485] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11107.236819] LustreError: 322838:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11107.510937] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11114.982193] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11123.437618] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11131.680264] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11131.778687] Lustre: lustre-OST0001: new disk, initializing [11131.781571] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11131.786207] Lustre: 323855:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11131.868635] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11139.460404] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11140.861684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11140.903362] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11141.041278] LustreError: 324277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11141.070845] LustreError: 324277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11142.054640] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11143.163644] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11143.178238] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11145.248992] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11145.267549] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11149.347085] LustreError: 324277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11149.370268] LustreError: 324277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11150.898459] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11151.152980] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11157.611520] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11157.637886] LustreError: 324269:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11159.288272] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0c890221800.ost_server_uuid 50 [11162.092675] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0c890221800.ost_server_uuid in FULL state after 0 sec [11171.859612] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0c890221800.ost_server_uuid 50 [11174.245419] LustreError: 324277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11174.290211] LustreError: 324277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [11174.442572] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0c890221800.ost_server_uuid in IDLE state after 0 sec [11178.598640] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11193.829676] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11193.833166] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11193.842591] LustreError: Skipped 2 previous similar messages [11193.844384] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11193.844395] Lustre: Skipped 1 previous similar message [11193.857669] Lustre: Skipped 1 previous similar message [11199.467696] Lustre: server umount lustre-OST0001 complete [11202.329481] LustreError: 322840:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11207.654688] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11212.538715] Lustre: server umount lustre-OST0000 complete [11217.900773] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11217.916843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11217.918385] Lustre: Skipped 2 previous similar messages [11217.940100] Lustre: Skipped 7 previous similar messages [11223.009939] LustreError: 322259:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11223.034919] LustreError: 322259:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11223.062823] Lustre: server umount lustre-MDT0000 complete [11227.840688] LustreError: 323854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788455301 with bad export cookie 8801093179922150059 [11227.853078] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11227.857438] LustreError: 323854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11228.230768] Lustre: server umount lustre-MDT0001 complete [11238.432199] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [11241.262606] Key type lgssc unregistered [11241.640466] LNet: 326551:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11241.653863] LNetError: 326551:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11241.672718] LNet: Removed LNI 192.168.204.123@tcp [11242.774569] Key type .llcrypt unregistered [11242.776364] Key type ._llcrypt unregistered [11267.753513] Key type ._llcrypt registered [11267.756376] Key type .llcrypt registered [11267.932336] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [11269.219726] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11269.320097] alg: No test for adler32 (adler32-zlib) [11270.339170] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11270.613890] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [11272.343308] Key type lgssc registered [11273.417843] Lustre: Echo OBD driver; http://www.lustre.org/ [11282.689292] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11282.733644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11284.261104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11289.160261] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11293.666091] LustreError: 327974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11293.704278] LustreError: 327974:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11296.752155] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11298.805658] LustreError: 327975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11303.913442] LustreError: 327974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11304.810654] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11305.309039] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11310.218511] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11322.029620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11322.525572] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11328.684285] LustreError: 329655:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11328.705643] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11329.018450] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11335.174430] LustreError: 329977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11335.215692] LustreError: 329977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11337.854700] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11345.379840] LustreError: 329654:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11345.424285] LustreError: 329654:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11345.930539] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11346.107884] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11351.579892] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11353.298382] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11362.975791] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11371.501755] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11371.951180] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11489.762070] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11489.768139] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11489.805197] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11489.832653] Lustre: Skipped 1 previous similar message [11492.457862] Lustre: server umount lustre-OST0001 complete [11494.891318] LustreError: 329656:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11494.925178] LustreError: 329656:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11500.009140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11500.017850] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11500.019582] Lustre: Skipped 2 previous similar messages [11500.026951] Lustre: Skipped 1 previous similar message [11503.085558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11508.195856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11508.213957] Lustre: Skipped 2 previous similar messages [11513.324539] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11513.328187] Lustre: Skipped 1 previous similar message [11514.335302] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11514.596826] Lustre: server umount lustre-OST0000 complete [11518.450494] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11523.553528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11523.566158] Lustre: Skipped 3 previous similar messages [11524.884811] Lustre: server umount lustre-MDT0000 complete [11528.678628] LustreError: 327975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11528.703799] LustreError: 327975:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [11529.409173] LustreError: 327954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788455603 with bad export cookie 3465667910926632485 [11529.417321] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11529.422396] LustreError: 327954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11529.842840] Lustre: server umount lustre-MDT0001 complete [11543.239352] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [11546.651368] Key type lgssc unregistered [11547.057247] LNet: 332926:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11547.078930] LNetError: 332926:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11547.095176] LNet: Removed LNI 192.168.204.123@tcp [11548.131960] Key type .llcrypt unregistered [11548.138332] Key type ._llcrypt unregistered [11610.791683] Key type ._llcrypt registered [11610.795591] Key type .llcrypt registered [11610.971942] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [11612.893924] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11613.008435] alg: No test for adler32 (adler32-zlib) [11614.007904] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11614.250396] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [11615.905845] Key type lgssc registered [11617.779886] Lustre: Echo OBD driver; http://www.lustre.org/ [11630.830417] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11630.844383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11632.523245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11637.760192] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11642.857259] LustreError: 335222:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11642.890458] LustreError: 335222:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11647.117381] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11647.516444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11652.267118] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11655.241418] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11662.666395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11663.038344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11669.883343] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11673.255873] LustreError: 336709:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11677.358646] LustreError: 337213:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11677.389948] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11677.401918] LustreError: 337213:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11679.641518] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11679.870697] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11685.392232] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11687.623289] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11697.545658] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11705.420164] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11716.070715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11716.075496] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11716.086263] Lustre: Skipped 2 previous similar messages [11716.108883] Lustre: Skipped 3 previous similar messages [11721.202520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11721.619869] Lustre: server umount lustre-MDT0000 complete [11725.293743] LustreError: 337535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788455799 with bad export cookie 4560429189005567100 [11725.311230] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11725.312098] LustreError: 337535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11725.663185] Lustre: server umount lustre-MDT0001 complete [11740.027887] Lustre: server umount lustre-OST0000 complete [11752.556770] Lustre: server umount lustre-OST0001 complete [11773.829754] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [11787.786195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11788.091233] LustreError: 340487:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11788.166462] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11793.376896] LustreError: 340488:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11793.385473] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11802.594218] LustreError: 340488:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11802.612485] LustreError: 340488:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11803.170374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11803.824770] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11808.294387] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11811.915124] Lustre: 341629:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11817.874473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11824.335927] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11828.458531] LustreError: 341983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11831.904216] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11832.195966] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11832.205414] Lustre: Skipped 1 previous similar message [11833.198389] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11833.213835] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11838.047881] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [11845.140966] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11848.594039] Lustre: 343500:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11853.795680] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11853.799597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11853.825017] Lustre: Skipped 4 previous similar messages [11853.835215] Lustre: Skipped 6 previous similar messages [11858.927118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11858.948927] Lustre: Skipped 3 previous similar messages [11859.667547] Lustre: server umount lustre-MDT0000 complete [11864.051436] LustreError: 343604:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11864.092884] LustreError: 343604:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11864.205568] LustreError: 341631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788455937 with bad export cookie 4560429189005568808 [11864.207227] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11864.223698] LustreError: 341631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11864.628570] Lustre: server umount lustre-MDT0001 complete [11879.607917] Lustre: server umount lustre-OST0000 complete [11894.566114] Lustre: server umount lustre-OST0001 complete [11899.767703] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11901.843120] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11904.982228] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:19:35 (1788455975) [11932.391175] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [11942.876512] loop0: detected capacity change from 0 to 1099511627776 [11943.610956] SGI XFS with ACLs, security attributes, quota, no debug enabled [11943.627752] *** VALIDATE xfs *** [11943.665020] XFS (loop0): Mounting V5 Filesystem [11943.761499] XFS (loop0): Ending clean mount [11945.309858] loop1: detected capacity change from 0 to 18691697672192 [12066.576324] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [12077.520667] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [12119.593409] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12119.963537] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12119.986364] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12120.040273] Lustre: lustre-MDT0000: new disk, initializing [12120.120519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12120.139697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12125.291167] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12137.642567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12137.798533] Lustre: 348644:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12137.940903] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12137.947499] Lustre: Skipped 1 previous similar message [12138.077468] Lustre: lustre-MDT0001: new disk, initializing [12138.262872] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12138.284921] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12142.915496] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12152.193340] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12158.708577] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12166.733820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12167.243652] Lustre: lustre-OST0000: new disk, initializing [12167.257122] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12167.267637] Lustre: 350614:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12169.313585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12169.322832] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12169.456267] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12175.255523] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12185.081969] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12190.491332] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12190.686734] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12195.213845] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12195.488934] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12200.419558] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12200.420592] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12200.438968] LustreError: Skipped 1 previous similar message [12200.445957] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12200.468925] Lustre: Skipped 1 previous similar message [12203.745807] Lustre: server umount lustre-OST0000 complete [12210.144620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12210.150518] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12210.164735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12210.171699] Lustre: Skipped 1 previous similar message [12214.800221] Lustre: server umount lustre-MDT0000 complete [12215.795527] LustreError: 349495:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12215.812602] LustreError: 349495:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12219.412294] LustreError: 348635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788456293 with bad export cookie 4560429189005570362 [12219.423596] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12219.431679] LustreError: 348635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12219.944770] Lustre: server umount lustre-MDT0001 complete [12232.095303] XFS (loop0): Unmounting Filesystem [12236.835489] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:25:08 (1788456308) [12244.681785] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12245.328852] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12245.338335] Lustre: Skipped 2 previous similar messages [12249.985641] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12260.250527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12267.342411] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12277.903758] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12286.622487] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12295.787612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12303.704707] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12313.349544] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12330.475957] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12330.485638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12330.486342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12330.486348] Lustre: Skipped 1 previous similar message [12330.487178] Lustre: Skipped 2 previous similar messages [12335.783919] Lustre: server umount lustre-OST0000 complete [12340.713726] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12346.774111] Lustre: server umount lustre-MDT0000 complete [12350.944674] LustreError: 353170:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12350.979651] LustreError: 353170:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12351.690934] LustreError: 353155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788456425 with bad export cookie 4560429189005571237 [12351.698102] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12351.709806] LustreError: 353155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12352.182361] Lustre: server umount lustre-MDT0001 complete [12365.802287] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [12369.115811] Key type lgssc unregistered [12369.444703] LNet: 356810:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12369.451295] LNetError: 356810:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12369.469300] LNet: Removed LNI 192.168.204.123@tcp [12370.562197] Key type .llcrypt unregistered [12370.564739] Key type ._llcrypt unregistered [12403.714156] Key type ._llcrypt registered [12403.716847] Key type .llcrypt registered [12403.894736] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [12405.415519] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12405.593931] alg: No test for adler32 (adler32-zlib) [12407.373608] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [12408.016116] LNet: Added LNI 192.168.204.123@tcp [8/256/0/180] [12409.919362] Key type lgssc registered [12412.745543] Lustre: Echo OBD driver; http://www.lustre.org/ [12424.997489] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12425.044865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12426.576044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12433.303580] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12436.969102] LustreError: 358376:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12437.000739] LustreError: 358376:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12442.081031] LustreError: 358377:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12445.687075] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12446.216064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12453.227400] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12463.525295] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12471.865483] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12482.557219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12483.001985] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12489.960711] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12498.837557] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12507.618321] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12507.641880] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12507.663920] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12507.689173] Lustre: Skipped 1 previous similar message [12511.049301] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.23@tcp (stopping) [12512.753118] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12512.766485] Lustre: Skipped 1 previous similar message [12513.711527] Lustre: server umount lustre-MDT0001 complete [12516.123950] LustreError: 358373:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12521.239518] LustreError: 358373:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12521.275493] LustreError: 358373:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12526.375365] LustreError: 359695:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12526.419577] LustreError: 359695:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12529.068354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12529.286291] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12529.293524] Lustre: Found index 1 for lustre-MDT0001, updating log [12529.411829] Lustre: 358365:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12529.853921] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12535.313933] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12535.841166] LustreError: 361795:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12535.887179] LustreError: 361795:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12535.910855] LustreError: 358368:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12536.114327] LustreError: 361804:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.123@tcp: cfg command failed: rc = -17 [12536.200119] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12536.200119] [12536.904506] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12550.643662] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12550.644057] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12550.666063] Lustre: Skipped 1 previous similar message [12550.667943] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12555.746305] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12555.767919] Lustre: Skipped 3 previous similar messages [12556.211459] Lustre: server umount lustre-MDT0001 complete [12557.078706] LustreError: 358372:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12557.115866] LustreError: 358372:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12571.243120] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12571.489430] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12571.508440] Lustre: Found index 1 for lustre-MDT0001, updating log [12571.611720] Lustre: 358365:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12572.285320] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12577.272851] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12579.230332] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12579.338670] LustreError: 362909:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12579.353141] LustreError: 362909:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12579.366198] LustreError: 358368:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12579.400336] LustreError: 358368:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12579.546858] LustreError: 362918:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12579.555972] LustreError: 362918:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12579.563628] LustreError: 362918:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.123@tcp: cfg command failed: rc = -17 [12579.585103] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.123@tcp [12579.585103] [12889.573254] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12889.582686] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12889.609469] Lustre: Skipped 1 previous similar message [12889.622919] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12889.632054] Lustre: Skipped 1 previous similar message [12892.189914] Lustre: server umount lustre-MDT0001 complete [12894.697093] LustreError: 359298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12894.716734] LustreError: 359298:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [12905.528554] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12905.663391] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12905.672633] Lustre: Found index 1 for lustre-MDT0001, updating log [12905.711641] Lustre: 358365:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12906.094063] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12911.080979] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [12911.087785] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12915.184188] LustreError: 370582:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12915.189510] LustreError: 370582:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12915.201689] LustreError: 358368:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12915.208383] LustreError: 358368:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12915.285280] LustreError: 370585:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12915.298320] LustreError: 370585:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.123@tcp: cfg command failed: rc = -114 [12915.305912] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12915.305912] [13233.637619] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13233.642864] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13233.656104] Lustre: Skipped 1 previous similar message [13233.658109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13233.658116] Lustre: Skipped 1 previous similar message [13239.557453] Lustre: server umount lustre-MDT0000 complete [13243.506118] LustreError: 358357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788457317 with bad export cookie 12302788412325260801 [13243.520256] LustreError: 358357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13243.521610] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13243.872338] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13243.873746] LustreError: 358371:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13243.883248] Lustre: Skipped 1 previous similar message [13243.893928] LustreError: 358371:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [13250.151879] Lustre: server umount lustre-MDT0001 complete [13255.690487] Lustre: server umount lustre-OST0000 complete [13270.047653] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [13280.734359] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [13320.542744] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 13:43:12 (1788457392) [13327.495211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13327.682127] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13327.710431] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13327.766824] Lustre: lustre-MDT0000: new disk, initializing [13327.828368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13327.844795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13332.616719] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [13344.680724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13344.843949] Lustre: 381015:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13344.916639] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13344.920115] Lustre: Skipped 1 previous similar message [13344.989319] Lustre: lustre-MDT0001: new disk, initializing [13345.084495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13345.119166] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13345.127948] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13350.682954] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [13362.225060] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13370.565514] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13380.518575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13381.108781] Lustre: lustre-OST0000: new disk, initializing [13381.115818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13381.136615] Lustre: 382988:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13381.292843] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13387.363242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13387.388356] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13387.531049] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13389.457627] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_default_debug -1 all [13398.899646] Lustre: DEBUG MARKER: oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13407.486156] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13407.492574] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13407.503710] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13407.510916] Lustre: Skipped 7 previous similar messages [13407.680594] Lustre: server umount lustre-MDT0000 complete [13408.238036] LustreError: 382581:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13408.273629] LustreError: 382581:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13412.278388] LustreError: 381825:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13412.282276] LustreError: 381008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788457485 with bad export cookie 12302788412325262754 [13412.292508] LustreError: 381825:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000004474b56 x1875332543335936/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [13412.301447] LustreError: MGC192.168.204.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13412.349339] LustreError: 381008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13412.625227] Lustre: server umount lustre-MDT0001 complete [13427.375652] Lustre: server umount lustre-OST0000 complete [13459.331954] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing set_hostid [13472.411752] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing load_modules_local [13509.746481] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13177 sec ========= 13:46:21 (1788457581) [13511.717836] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 13:46:23 (1788457583) === [13515.472375] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 13:46:27 (1788457587) === [13559.608103] Lustre: DEBUG MARKER: oleg423-server.virtnet: executing unload_modules_local [13562.722409] Key type lgssc unregistered [13563.024823] LNet: 389359:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13563.031843] LNetError: 389359:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13563.049071] LNet: Removed LNI 192.168.204.123@tcp [13564.141828] Key type .llcrypt unregistered [13564.150434] Key type ._llcrypt unregistered