[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 593220477 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001019] APIC: Switch to symmetric I/O mode setup [ 0.002542] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007841] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008029] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009011] pid_max: default: 32768 minimum: 301 [ 0.010188] LSM: Security Framework initializing [ 0.011000] Yama: becoming mindful. [ 0.011037] SELinux: Initializing. [ 0.012073] *** VALIDATE selinux *** [ 0.025856] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.031524] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.033075] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.034098] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.035162] *** VALIDATE tmpfs *** [ 0.037277] *** VALIDATE proc *** [ 0.038244] *** VALIDATE cgroup *** [ 0.039008] *** VALIDATE cgroup2 *** [ 0.040253] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.041148] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.042006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.043029] Spectre V2 : User space: Vulnerable [ 0.044006] Speculative Store Bypass: Vulnerable [ 0.047654] debug: unmapping init [mem 0xffffffffa6459000-0xffffffffa6460fff] [ 0.049268] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050690] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051019] ... version: 2 [ 0.051946] ... bit width: 48 [ 0.052013] ... generic registers: 4 [ 0.052940] ... value mask: 0000ffffffffffff [ 0.053014] ... max period: 00007fffffffffff [ 0.054011] ... fixed-purpose events: 3 [ 0.055009] ... event mask: 000000070000000f [ 0.057325] rcu: Hierarchical SRCU implementation. [ 0.059604] smp: Bringing up secondary CPUs ... [ 0.060635] x86: Booting SMP configuration: [ 0.061018] .... node #0, CPUs: #1 #2 #3 [ 0.069068] smp: Brought up 1 node, 4 CPUs [ 0.071010] smpboot: Max logical packages: 1 [ 0.072009] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.115172] node 0 deferred pages initialised in 41ms [ 0.168121] devtmpfs: initialized [ 0.170552] x86/mm: Memory block size: 128MB [ 0.174868] gcov: version magic: 0x41383552 [ 0.179124] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.181121] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.187075] pinctrl core: initialized pinctrl subsystem [ 0.189524] [ 0.191008] ************************************************************* [ 0.193015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.196012] ** ** [ 0.198010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.200012] ** ** [ 0.202156] ** This means that this kernel is built to expose internal ** [ 0.204010] ** IOMMU data structures, which may compromise security on ** [ 0.206011] ** your system. ** [ 0.208011] ** ** [ 0.210009] ** If you see this message and you are not debugging the ** [ 0.213011] ** kernel, report this immediately to your vendor! ** [ 0.215008] ** ** [ 0.218011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.221011] ************************************************************* [ 0.224285] NET: Registered protocol family 16 [ 0.226798] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.229052] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.231075] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.235595] cpuidle: using governor menu [ 0.239041] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.245077] PCI: Using configuration type 1 for base access [ 0.247157] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.256176] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.257015] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.259198] cryptd: max_cpu_qlen set to 1000 [ 0.262525] ACPI: Added _OSI(Module Device) [ 0.265015] ACPI: Added _OSI(Processor Device) [ 0.267012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.269015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.275176] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.283950] ACPI: Interpreter enabled [ 0.285055] ACPI: PM: (supports S0 S3 S4 S5) [ 0.286010] ACPI: Using IOAPIC for interrupt routing [ 0.288162] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.295836] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.311030] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.314038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.316016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.320285] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.328296] acpiphp: Slot [2] registered [ 0.330737] acpiphp: Slot [5] registered [ 0.332261] acpiphp: Slot [6] registered [ 0.335357] acpiphp: Slot [7] registered [ 0.336109] acpiphp: Slot [8] registered [ 0.338094] acpiphp: Slot [9] registered [ 0.340151] acpiphp: Slot [10] registered [ 0.341233] acpiphp: Slot [3] registered [ 0.342105] acpiphp: Slot [4] registered [ 0.344201] acpiphp: Slot [11] registered [ 0.346088] acpiphp: Slot [12] registered [ 0.347212] acpiphp: Slot [13] registered [ 0.349089] acpiphp: Slot [14] registered [ 0.350200] acpiphp: Slot [15] registered [ 0.352108] acpiphp: Slot [16] registered [ 0.353086] acpiphp: Slot [17] registered [ 0.355153] acpiphp: Slot [18] registered [ 0.357174] acpiphp: Slot [19] registered [ 0.359245] acpiphp: Slot [20] registered [ 0.362243] acpiphp: Slot [21] registered [ 0.363293] acpiphp: Slot [22] registered [ 0.365085] acpiphp: Slot [23] registered [ 0.366110] acpiphp: Slot [24] registered [ 0.367233] acpiphp: Slot [25] registered [ 0.369120] acpiphp: Slot [26] registered [ 0.371156] acpiphp: Slot [27] registered [ 0.372127] acpiphp: Slot [28] registered [ 0.374139] acpiphp: Slot [29] registered [ 0.376130] acpiphp: Slot [30] registered [ 0.377431] acpiphp: Slot [31] registered [ 0.380114] PCI host bridge to bus 0000:00 [ 0.382025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.384189] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.387025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.390028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.393032] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.396029] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.398195] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.403000] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.406763] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.420017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.427019] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.430020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.433023] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.438024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.440263] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.443779] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.446041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.449085] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.453027] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.467025] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.472019] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.480810] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.492021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.501019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.520020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.535140] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.543019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.550022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.585000] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.621000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.630022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.637023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.654017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.664629] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.669015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.675015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.689028] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.697000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.707020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.709032] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.726019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.739000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.763014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.771020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.786018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.792000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.793397] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.794347] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.795556] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.796087] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.802110] iommu: Default domain type: Passthrough [ 0.806398] SCSI subsystem initialized [ 0.809169] ACPI: bus type USB registered [ 0.811107] usbcore: registered new interface driver usbfs [ 0.812065] usbcore: registered new interface driver hub [ 0.814093] usbcore: registered new device driver usb [ 0.816178] pps_core: LinuxPPS API ver. 1 registered [ 0.818013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.820059] PTP clock support registered [ 0.822237] EDAC MC: Ver: 3.0.0 [ 0.824017] PCI: Using ACPI for IRQ routing [ 0.825568] NetLabel: Initializing [ 0.826426] NetLabel: domain hash size = 128 [ 0.828011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.829167] NetLabel: unlabeled traffic allowed by default [ 0.831360] vgaarb: loaded [ 0.832351] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.834010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.839515] clocksource: Switched to clocksource kvm-clock [ 0.976391] VFS: Disk quotas dquot_6.6.0 [ 0.978215] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.981138] *** VALIDATE ramfs *** [ 0.982447] *** VALIDATE hugetlbfs *** [ 0.984873] pnp: PnP ACPI init [ 0.986845] pnp: PnP ACPI: found 6 devices [ 1.004642] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.008252] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.010448] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.012632] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.014406] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.016282] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.018670] NET: Registered protocol family 2 [ 1.021249] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.025911] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.028596] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.033542] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.036421] TCP: Hash tables configured (established 65536 bind 65536) [ 1.039488] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.042593] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.045546] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.048416] NET: Registered protocol family 1 [ 1.051369] RPC: Registered named UNIX socket transport module. [ 1.053522] RPC: Registered udp transport module. [ 1.057072] RPC: Registered tcp transport module. [ 1.058806] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.061528] NET: Registered protocol family 44 [ 1.062901] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.064630] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.066303] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.067782] PCI: CLS 0 bytes, default 64 [ 1.069231] Unpacking initramfs... [ 2.788720] debug: unmapping init [mem 0xffff9995bcc54000-0xffff9995bffbffff] [ 2.793222] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.794934] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.797347] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.303823] Initialise system trusted keyrings [ 3.305223] Key type blacklist registered [ 3.306540] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.315429] zbud: loaded [ 3.318120] *** VALIDATE nfs *** [ 3.318882] *** VALIDATE nfs4 *** [ 3.320076] pstore: using deflate compression [ 3.323830] Platform Keyring initialized [ 3.440342] NET: Registered protocol family 38 [ 3.442134] Key type asymmetric registered [ 3.443377] Asymmetric key parser 'x509' registered [ 3.444741] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.446893] io scheduler mq-deadline registered [ 3.448010] io scheduler kyber registered [ 3.448947] io scheduler bfq registered [ 3.450442] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.452362] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.454536] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.457113] ACPI: Power Button [PWRF] [ 3.465611] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.475865] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.490785] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.500922] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.514407] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.544035] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.571844] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.575625] Non-volatile memory driver v1.3 [ 3.576929] Linux agpgart interface v0.103 [ 3.602417] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.604560] vda: detected capacity change from 0 to 74698752 [ 3.617102] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.619264] vdb: detected capacity change from 0 to 1073741824 [ 3.633057] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.637145] vdc: detected capacity change from 0 to 2621440000 [ 3.653111] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.655719] vdd: detected capacity change from 0 to 2621440000 [ 3.672237] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.675142] vde: detected capacity change from 0 to 4294967296 [ 3.707782] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.756977] vdf: detected capacity change from 0 to 4294967296 [ 3.765828] libphy: Fixed MDIO Bus: probed [ 3.779433] usbcore: registered new interface driver usbserial_generic [ 3.801478] usbserial: USB Serial support registered for generic [ 3.804894] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.811693] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.814465] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.817838] mousedev: PS/2 mouse device common for all mice [ 3.821184] rtc_cmos 00:05: RTC can wake from S4 [ 3.825437] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.828418] rtc_cmos 00:05: registered as rtc0 [ 3.831531] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.834281] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.835307] intel_pstate: CPU model not supported [ 3.839223] hid: raw HID events driver (C) Jiri Kosina [ 3.843388] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.844636] usbcore: registered new interface driver usbhid [ 3.844641] usbhid: USB HID core driver [ 3.844758] drop_monitor: Initializing network drop monitor service [ 3.844960] Initializing XFRM netlink socket [ 3.846318] NET: Registered protocol family 10 [ 3.863312] Segment Routing with IPv6 [ 3.864905] NET: Registered protocol family 17 [ 3.866890] mpls_gso: MPLS GSO support [ 3.872823] RAS: Correctable Errors collector initialized. [ 3.874718] AVX version of gcm_enc/dec engaged. [ 3.876498] AES CTR mode by8 optimization enabled [ 3.957825] sched_clock: Marking stable (3957747559, 0)->(4888133137, -930385578) [ 3.961699] registered taskstats version 1 [ 3.963774] Loading compiled-in X.509 certificates [ 3.966284] zswap: loaded using pool lzo/zbud [ 3.998181] Key type big_key registered [ 4.021555] Key type encrypted registered [ 4.023631] ima: No TPM chip found, activating TPM-bypass! [ 4.024671] ima: Allocated hash algorithm: sha1 [ 4.025637] ima: No architecture policies found [ 4.026756] evm: Initialising EVM extended attributes: [ 4.027928] evm: security.selinux [ 4.028679] evm: security.ima [ 4.029517] evm: security.capability [ 4.030407] evm: HMAC attrs: 0x1 [ 4.032611] rtc_cmos 00:05: setting system clock to 2026-08-08 08:23:20 UTC (1786177400) [ 4.038550] debug: unmapping init [mem 0xffffffffa7403000-0xffffffffa75fffff] [ 4.042162] debug: unmapping init [mem 0xffffffffa6182000-0xffffffffa6458fff] [ 4.050217] Write protecting the kernel read-only data: 28672k [ 4.054586] debug: unmapping init [mem 0xffffffffa4803000-0xffffffffa49fffff] [ 4.063261] debug: unmapping init [mem 0xffffffffa5114000-0xffffffffa51fffff] [ 4.121259] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.129504] systemd[1]: Detected virtualization kvm. [ 4.132566] systemd[1]: Detected architecture x86-64. [ 4.135747] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.176193] systemd[1]: No hostname configured. [ 4.179167] systemd[1]: Set hostname to . [ 4.181428] random: systemd: uninitialized urandom read (16 bytes read) [ 4.184210] systemd[1]: Initializing machine ID from random generator. [ 4.396526] random: systemd: uninitialized urandom read (16 bytes read) [ 4.399442] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.405274] random: systemd: uninitialized urandom read (16 bytes read) [ 4.408512] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.415390] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Journal Service... Starting Setup Virtual Console... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.960684] device-mapper: uevent: version 1.0.3 [ 5.962750] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [[ 7.021951] random: fast init done  OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 7.035337] virtio_net virtio0 ens2: renamed from eth0 [ 7.086644] scsi host0: ata_piix Starting dracut initqueue hook... [ 7.170185] scsi host1: ata_piix [ 7.332815] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.335967] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 12.697952] random: crng init done [ 12.699713] random: 7 urandom warning(s) missed due to ratelimiting [ 15.491773] dracut-initqueue[597]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 16.899619] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 18.966093] printk: systemd: 26 output lines suppressed due to ratelimiting [ 19.388234] SELinux: Disabled at runtime. [ 19.455862] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 19.463941] systemd[1]: Detected virtualization kvm. [ 19.465634] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 20.783618] systemd[1]: initrd-switch-root.service: Succeeded. [ 20.787608] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 20.794698] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 20.802281] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 20.807228] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 20.817929] systemd[1]: Starting Journal Service... Starting Journal Service... [ 20.848706] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. [ 21.089047] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started udev Coldplug all Devices. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 25.245991] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 29.466355] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 29.472164] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 30.833429] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 31.011934] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…only root support (13s / no limit) [** ] A start job is running for Configur…only root support (14s / no limit) [*** ] A start job is running for Configur…only root support (14s / no limit) [ *** ] A start job is running for Configur…only root support (15s / no limit)[ 36.280800] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (15s / no limit) [ ***] A start job is running for Configur…only root support (16s / no limit)[ 37.140812] NFS: Registering the id_resolver key type [ 37.143220] Key type id_resolver registered [ 37.145323] Key type id_legacy registered [ **] A start job is running for Configur…only root support (17s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg228-server login: [ 64.744364] hrtimer: interrupt took 7772476 ns [ 106.228886] libcfs: loading out-of-tree module taints kernel. [ 106.264810] Key type ._llcrypt registered [ 106.266780] Key type .llcrypt registered [ 106.357417] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 123.093581] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 124.800017] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 124.827262] alg: No test for adler32 (adler32-zlib) [ 126.340941] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 127.527393] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 129.448538] Key type lgssc registered [ 131.225596] Lustre: Echo OBD driver; http://www.lustre.org/ [ 147.529180] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 186.948948] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 200.566185] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 200.586868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 201.734251] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 201.767936] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 201.837279] Lustre: lustre-MDT0000: new disk, initializing [ 201.924919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 201.944553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 205.115949] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 217.161747] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 217.252863] Lustre: 6520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 217.286480] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 217.290568] Lustre: Skipped 1 previous similar message [ 217.363591] Lustre: lustre-MDT0001: new disk, initializing [ 217.403792] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 217.428232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 217.443183] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 221.355480] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 226.013584] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 235.651547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 235.905571] Lustre: lustre-OST0000: new disk, initializing [ 235.908595] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 235.913269] Lustre: 8458:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 235.973121] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 241.857747] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.745538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 244.750247] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 244.845807] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 257.135948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 257.257243] Lustre: lustre-OST0001: new disk, initializing [ 257.260275] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 257.265223] Lustre: 9532:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 257.332139] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 263.751228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 263.760951] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 263.854964] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 264.046053] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 276.407175] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 287.798144] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 294.495457] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing check_logdir /tmp/testlogs/ [ 299.565329] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing yml_node [ 303.932878] Lustre: DEBUG MARKER: Client: 2.17.56.50 [ 306.130790] Lustre: DEBUG MARKER: MDS: 2.17.56.50 [ 308.362408] Lustre: DEBUG MARKER: OSS: 2.17.56.50 [ 309.812618] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 8 04:28:25 EDT 2026 [ 324.977814] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 326.256851] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 345.568094] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 345.576568] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 345.600195] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 346.080215] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 346.103411] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 346.107149] Lustre: Skipped 3 previous similar messages [ 349.656352] Lustre: server umount lustre-MDT0000 complete [ 352.710504] LustreError: 6514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786177749 with bad export cookie 6023267311225733833 [ 352.710750] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 352.719162] LustreError: 6514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 356.320187] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 356.320702] LustreError: 6529:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 356.332106] Lustre: Skipped 2 previous similar messages [ 356.333217] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 356.346150] LustreError: 6529:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 6 previous similar messages [ 357.345219] LustreError: 6528:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 357.359495] LustreError: 6528:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 362.463874] LustreError: 6530:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 362.463970] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 362.478026] LustreError: 6530:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 362.493735] Lustre: Skipped 4 previous similar messages [ 367.071548] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 367.229633] Lustre: server umount lustre-MDT0001 complete [ 371.377288] Lustre: server umount lustre-OST0000 complete [ 376.234661] Lustre: server umount lustre-OST0001 complete [ 383.319687] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 394.759149] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 432.132503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 432.389588] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 432.428869] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 432.560423] Lustre: lustre-MDT0000: new disk, initializing [ 432.652089] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 432.685236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 437.346750] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 448.077461] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 448.229560] Lustre: 15087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 448.311762] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 448.318589] Lustre: Skipped 1 previous similar message [ 448.405423] Lustre: lustre-MDT0001: new disk, initializing [ 448.508485] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 448.563315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 448.592114] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 452.649412] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 463.163432] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 470.162687] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 476.589569] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 476.969609] Lustre: lustre-OST0000: new disk, initializing [ 476.976228] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 476.985049] Lustre: 17054:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 477.153124] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 478.349841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 478.371489] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 478.491469] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 483.729717] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 492.978931] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 497.108888] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 497.251395] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 501.313140] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 501.522567] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 504.303609] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 504.304781] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 504.311770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 504.311779] Lustre: Skipped 2 previous similar messages [ 504.327659] LustreError: Skipped 1 previous similar message [ 504.399615] Lustre: Skipped 2 previous similar messages [ 509.450293] LustreError: 17038:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.464523] LustreError: 17038:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 509.591946] Lustre: server umount lustre-OST0000 complete [ 514.562180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 514.592602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 514.603194] Lustre: Skipped 1 previous similar message [ 519.660279] LustreError: 15094:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.692968] LustreError: 15094:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 519.918373] Lustre: server umount lustre-MDT0000 complete [ 523.835888] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786177920 with bad export cookie 6023267311225735366 [ 523.840217] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 523.847600] LustreError: 15081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 524.442122] Lustre: server umount lustre-MDT0001 complete [ 530.572374] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:32:05 (1786177925) [ 537.356965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 537.629806] LustreError: 19317:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.691669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 542.297468] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 553.318717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 558.931432] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 565.411953] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 570.088611] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 577.065460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 584.189745] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 592.741735] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 596.374383] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 604.133117] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 604.145517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 604.166803] Lustre: Skipped 2 previous similar messages [ 604.196122] Lustre: Skipped 2 previous similar messages [ 607.929597] Lustre: server umount lustre-MDT0000 complete [ 609.252905] LustreError: 19314:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 609.285044] LustreError: 19314:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 5 previous similar messages [ 612.595862] LustreError: 19299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178009 with bad export cookie 6023267311225736248 [ 612.602352] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 612.604615] LustreError: 19299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 613.066904] Lustre: server umount lustre-MDT0001 complete [ 628.279548] Lustre: server umount lustre-OST0000 complete [ 653.814131] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 664.908752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 665.340876] LustreError: 23701:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 665.401529] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 665.408388] Lustre: Skipped 2 previous similar messages [ 669.846509] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 679.128353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 683.841856] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 686.574202] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 693.818752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 700.296242] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 708.648477] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 708.776835] Lustre: lustre-OST0001: new disk, initializing [ 708.779832] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 708.790713] Lustre: 26017:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 710.402771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 710.420539] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 710.490691] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 714.757602] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 723.477740] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 726.809754] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 740.836319] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 740.846339] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 740.860805] Lustre: Skipped 1 previous similar message [ 740.867989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 744.639962] Lustre: server umount lustre-MDT0000 complete [ 746.466834] LustreError: 24433:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 746.477950] LustreError: 24433:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 5 previous similar messages [ 748.372958] LustreError: 26016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178144 with bad export cookie 6023267311225737095 [ 748.379227] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 748.383506] LustreError: 26016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 748.638636] Lustre: server umount lustre-MDT0001 complete [ 763.143105] Lustre: server umount lustre-OST0000 complete [ 777.448190] Lustre: server umount lustre-OST0001 complete [ 795.491512] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 805.282904] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 805.872590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 805.891470] Lustre: Skipped 3 previous similar messages [ 809.780578] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 819.297676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 824.532838] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 827.659650] Lustre: 30188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 834.382314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 841.302271] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 850.124913] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 858.814079] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 866.837885] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 871.121594] Lustre: 32065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 877.546961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 877.562215] Lustre: Skipped 3 previous similar messages [ 877.566118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 877.592613] Lustre: Skipped 7 previous similar messages [ 883.223296] Lustre: server umount lustre-MDT0000 complete [ 887.283672] LustreError: 29021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178283 with bad export cookie 6023267311225738383 [ 887.287952] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 887.298481] LustreError: 29021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 887.783464] LustreError: 29035:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 887.794021] LustreError: 29035:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 7 previous similar messages [ 887.895964] Lustre: server umount lustre-MDT0001 complete [ 894.100975] Lustre: server umount lustre-OST0000 complete [ 897.815728] Lustre: server umount lustre-OST0001 complete [ 907.292780] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:38:22 (1786178302) [ 933.329684] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 943.191434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 947.620480] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 955.229545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 959.280525] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 962.648654] Lustre: 36013:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 969.769677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 975.360692] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 985.398352] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 990.828443] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 998.029774] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1001.761382] Lustre: 37896:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1007.085360] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1007.095797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1007.113991] Lustre: Skipped 3 previous similar messages [ 1007.126181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1007.143732] Lustre: Skipped 4 previous similar messages [ 1013.330376] Lustre: server umount lustre-MDT0000 complete [ 1017.060454] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178413 with bad export cookie 6023267311225739496 [ 1017.073314] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1017.077626] LustreError: 34847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1017.354918] Lustre: server umount lustre-MDT0001 complete [ 1031.626806] Lustre: server umount lustre-OST0000 complete [ 1035.745986] Lustre: 3656:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786178416/real 1786178416] req@ffff999605072d80 x1872942882694912/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786178432 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1036.016843] Lustre: server umount lustre-OST0001 complete [ 1047.678385] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:40:42 (1786178442) [ 1065.771517] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 1077.951282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1078.553398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1078.558417] Lustre: Skipped 7 previous similar messages [ 1083.416221] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1092.851893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1099.902876] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1103.882679] Lustre: 41187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1111.948622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1120.268736] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1131.319839] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1140.958382] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1149.911153] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1154.120245] Lustre: 43068:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1155.323411] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1155.328455] Lustre: Skipped 1 previous similar message [ 1160.161120] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1160.186662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1160.210618] Lustre: Skipped 4 previous similar messages [ 1165.293128] Lustre: server umount lustre-MDT0000 complete [ 1165.300392] LustreError: 41150:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1165.319765] LustreError: 41150:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 17 previous similar messages [ 1169.718069] LustreError: 40017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178566 with bad export cookie 6023267311225740602 [ 1169.721692] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1169.726632] LustreError: 40017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1184.223216] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1184.469802] Lustre: server umount lustre-MDT0001 complete [ 1193.475670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1195.679337] LustreError: 43932:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 1195.691114] LustreError: 43932:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff999630bb0a80 x1872942882752640/t0(0) o250->MGC192.168.202.128@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1786178592 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1195.730742] LustreError: 43932:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 1202.365809] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1210.876538] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1212.419385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1218.037264] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1218.042709] Lustre: Skipped 1 previous similar message [ 1218.149308] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1226.425136] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1228.070863] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1233.559188] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1235.056818] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1246.741182] Lustre: server umount lustre-MDT0000 complete [ 1249.853608] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178646 with bad export cookie 6023267311225741722 [ 1249.854463] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1249.860748] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1250.126922] Lustre: server umount lustre-MDT0001 complete [ 1264.290650] Lustre: server umount lustre-OST0000 complete [ 1277.943881] Lustre: server umount lustre-OST0001 complete [ 1288.106206] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:44:43 (1786178683) [ 1308.525721] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 1319.249678] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1323.964385] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1332.808520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1336.189698] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1339.087979] Lustre: 48831:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1344.293844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1350.357705] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1358.991198] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1365.531670] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1372.910077] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1376.908973] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1389.537718] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1389.552298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1389.554780] Lustre: Skipped 27 previous similar messages [ 1390.173181] Lustre: server umount lustre-MDT0000 complete [ 1393.757446] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786178790 with bad export cookie 6023267311225742548 [ 1393.765318] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1393.768282] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1394.089780] Lustre: server umount lustre-MDT0001 complete [ 1408.340575] Lustre: server umount lustre-OST0000 complete [ 1410.533086] Lustre: 3657:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786178791/real 1786178791] req@ffff999503c04000 x1872942882840704/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786178807 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1412.146564] Lustre: server umount lustre-OST0001 complete [ 1420.417783] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:46:56 (1786178816) [ 1422.880956] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1424.679629] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:46:59 (1786178819) [ 1435.001342] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 1442.219215] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 1463.344025] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1463.628126] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1463.724259] Lustre: lustre-MDT0000: new disk, initializing [ 1463.817710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1468.638551] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1478.331709] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1485.558738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1485.882410] Lustre: lustre-OST0000: new disk, initializing [ 1485.886582] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1485.895320] Lustre: Skipped 1 previous similar message [ 1485.900801] Lustre: 54909:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1487.387986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1487.394655] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1487.428245] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1492.269342] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1502.176326] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1504.385557] Lustre: DEBUG MARKER: create test files [ 1533.429903] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1533.432099] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1533.463577] Lustre: Skipped 14 previous similar messages [ 1546.207491] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1546.448984] Lustre: server umount lustre-OST0000 complete [ 1550.390954] Lustre: server umount lustre-MDT0000 complete [ 1564.619440] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 1568.365526] Key type lgssc unregistered [ 1568.698678] LNet: 56550:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1568.703938] LNetError: 56550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1569.765234] LNet: Removed LNI 192.168.202.128@tcp [ 1571.241545] Key type .llcrypt unregistered [ 1571.247419] Key type ._llcrypt unregistered [ 1605.729459] Key type ._llcrypt registered [ 1605.732032] Key type .llcrypt registered [ 1605.848739] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 1607.390987] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1607.427018] alg: No test for adler32 (adler32-zlib) [ 1608.423468] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 1608.681572] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 1610.327162] Key type lgssc registered [ 1611.230302] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1617.625765] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1617.646096] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1619.061465] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1622.850360] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1629.106942] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1636.141640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1636.663399] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1643.022988] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1643.817599] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:96 to 0x240000400:129) [ 1643.931310] Lustre: 59671:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1643.955223] Lustre: 59671:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1643.961718] Lustre: 59671:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1643.977510] Lustre: 59671:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1643.989596] Lustre: 59671:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1643.997686] Lustre: 59671:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1651.496799] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1658.182929] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1664.091596] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1674.742096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1674.757169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1674.778403] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1678.388057] Lustre: server umount lustre-OST0000 complete [ 1683.089597] Lustre: server umount lustre-MDT0000 complete [ 1695.618292] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 1698.819846] Key type lgssc unregistered [ 1699.161678] LNet: 60898:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1699.171474] LNetError: 60898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1699.186195] LNet: Removed LNI 192.168.202.128@tcp [ 1700.340734] Key type .llcrypt unregistered [ 1700.343780] Key type ._llcrypt unregistered [ 1729.174213] Key type ._llcrypt registered [ 1729.175580] Key type .llcrypt registered [ 1729.276372] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 1730.684446] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1730.710888] alg: No test for adler32 (adler32-zlib) [ 1731.842152] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 1732.063813] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 1733.735162] Key type lgssc registered [ 1734.714174] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1743.367942] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1743.399276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1744.950764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1749.750618] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1757.023868] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1763.530739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1763.869634] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1770.026145] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1770.128692] Lustre: 63726:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1770.137598] Lustre: 63726:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1770.141871] Lustre: 63726:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1770.146200] Lustre: 63726:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1770.153938] Lustre: 63726:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1770.160460] Lustre: 63726:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1770.455563] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1777.678818] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1782.341512] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1821.159320] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1821.160154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1821.186917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1825.905954] Lustre: server umount lustre-OST0000 complete [ 1830.015881] Lustre: server umount lustre-MDT0000 complete [ 1842.255667] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 1845.202219] Key type lgssc unregistered [ 1845.475128] LNet: 65312:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1845.488885] LNetError: 65312:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1845.509581] LNet: Removed LNI 192.168.202.128@tcp [ 1846.484908] Key type .llcrypt unregistered [ 1846.489678] Key type ._llcrypt unregistered [ 1863.942979] Key type ._llcrypt registered [ 1863.945258] Key type .llcrypt registered [ 1864.027598] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 1877.912888] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 1879.178292] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1879.212764] alg: No test for adler32 (adler32-zlib) [ 1880.231613] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 1880.470118] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 1882.191221] Key type lgssc registered [ 1883.668748] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1920.587725] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1920.619489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1921.937906] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1921.977292] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1922.104848] Lustre: lustre-MDT0000: new disk, initializing [ 1922.181084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1922.194868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1927.297285] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1938.129382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1938.213945] Lustre: 68464:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1938.274832] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1938.278548] Lustre: Skipped 1 previous similar message [ 1938.346410] Lustre: lustre-MDT0001: new disk, initializing [ 1938.445038] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1938.486355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1938.506149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1943.249581] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1953.625572] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1959.496373] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1966.648478] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1966.994569] Lustre: lustre-OST0000: new disk, initializing [ 1966.997368] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1967.001730] Lustre: 70435:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1967.080349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1971.845801] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 1976.383078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1976.394386] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1976.487879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1980.564665] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1985.281293] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1985.504910] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1990.176205] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1990.462770] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1997.281134] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1997.286282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1997.309365] Lustre: Skipped 1 previous similar message [ 1997.314972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1998.734385] Lustre: server umount lustre-OST0000 complete [ 2004.968900] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2004.984223] LustreError: Skipped 1 previous similar message [ 2005.001297] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2005.036167] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2005.062877] Lustre: Skipped 1 previous similar message [ 2007.523994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2007.525588] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2009.985134] Lustre: server umount lustre-MDT0000 complete [ 2015.814617] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786179412 with bad export cookie 4146636619790883214 [ 2015.830645] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2015.840608] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2016.459073] Lustre: server umount lustre-MDT0001 complete [ 2027.596742] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:57:02 (1786179422) [ 2039.249779] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 2080.761252] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 2090.062223] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 2124.832867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2125.168603] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2125.190311] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2125.264616] Lustre: lustre-MDT0000: new disk, initializing [ 2125.316467] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2125.326674] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2130.431840] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2141.600160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2141.759830] Lustre: 76456:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2141.860824] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2141.865346] Lustre: Skipped 1 previous similar message [ 2141.982316] Lustre: lustre-MDT0001: new disk, initializing [ 2142.100808] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2142.164407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2142.185378] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2145.654814] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2154.942815] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2161.207509] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2168.213686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2168.452769] Lustre: lustre-OST0000: new disk, initializing [ 2168.456828] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2168.462841] Lustre: 78426:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2168.557597] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2174.406649] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2174.523611] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2174.537134] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2174.654809] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2183.869806] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2188.761798] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2188.991624] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2192.165694] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2192.312948] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2195.428093] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2195.429648] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2195.432912] LustreError: Skipped 1 previous similar message [ 2195.434320] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2195.434326] Lustre: Skipped 1 previous similar message [ 2195.473197] Lustre: Skipped 1 previous similar message [ 2200.371689] Lustre: server umount lustre-OST0000 complete [ 2205.668628] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2205.684173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2205.694076] Lustre: Skipped 2 previous similar messages [ 2210.695697] Lustre: server umount lustre-MDT0000 complete [ 2210.786756] LustreError: 77398:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2210.813378] LustreError: 77398:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 4 previous similar messages [ 2213.940302] LustreError: 76448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786179610 with bad export cookie 4146636619790884089 [ 2213.949439] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2213.950515] LustreError: 76448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2214.390919] Lustre: server umount lustre-MDT0001 complete [ 2223.705297] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:00:18 (1786179618) [ 2230.899114] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2231.237676] LustreError: 80834:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2231.351625] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2236.340789] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2236.384291] LustreError: 80835:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2241.508492] LustreError: 80834:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2245.369964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2249.943900] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2258.151700] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2265.984216] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2273.616732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2273.919210] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2273.925068] Lustre: Skipped 1 previous similar message [ 2280.092228] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2288.068331] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2291.599883] Lustre: 83520:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting f26c2839-f2ae-4219-82aa-e45c54f82b37 at adminstrative request [ 2296.812912] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2296.833140] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2307.051240] Lustre: MGS: Client f26c2839-f2ae-4219-82aa-e45c54f82b37 (at 0@lo) reconnecting [ 2307.060776] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x398bd0092f43206b to 0x398bd0092f432312 [ 2307.082635] Lustre: MGC192.168.202.128@tcp: Connection restored to 0@lo (at 0@lo) [ 2330.086828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2330.326936] Lustre: lustre-OST0001: new disk, initializing [ 2330.331366] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2330.338963] Lustre: 83911:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2337.036720] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2340.570880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2340.595714] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2340.795256] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2348.839126] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2356.207714] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2356.217540] Lustre: Skipped 1 previous similar message [ 2356.225675] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2356.226432] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2358.667037] Lustre: server umount lustre-OST0001 complete [ 2361.318733] LustreError: 84769:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2361.381950] LustreError: 84769:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 2371.557812] LustreError: 84769:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2371.567744] LustreError: 84769:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [ 2376.679383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2376.684748] Lustre: Skipped 6 previous similar messages [ 2378.207203] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2378.514424] Lustre: server umount lustre-OST0000 complete [ 2383.841285] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2383.863146] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2383.883460] Lustre: Skipped 3 previous similar messages [ 2397.152365] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2397.317349] Lustre: server umount lustre-MDT0000 complete [ 2401.739413] LustreError: 80815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786179798 with bad export cookie 4146636619790885650 [ 2401.744086] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2401.754516] LustreError: 80815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2402.139949] Lustre: server umount lustre-MDT0001 complete [ 2410.976569] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:03:26 (1786179806) [ 2412.777711] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2414.654797] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:03:29 (1786179809) [ 2416.434154] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2418.220057] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:03:33 (1786179813) [ 2419.552474] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2420.954505] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:03:36 (1786179816) [ 2422.709122] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2424.518068] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:03:39 (1786179819) [ 2426.209893] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2432.279620] loop: module loaded [ 2432.306225] loop0: detected capacity change from 0 to 4294967296 [ 2432.369036] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2433.942455] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2437.203225] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2440.606878] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2468.415834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2468.452065] LustreError: 88041:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2468.459714] LustreError: 88041:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2468.468461] LustreError: 88041:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2468.477448] LustreError: 88041:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2487.244301] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 2498.947498] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 2539.926932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2540.147557] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2540.169699] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2540.253597] Lustre: lustre-MDT0000: new disk, initializing [ 2540.326819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2540.332059] Lustre: Skipped 1 previous similar message [ 2540.345686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2545.331733] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2554.802984] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2554.956237] Lustre: 90906:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2555.321921] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2559.935316] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2569.763857] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2574.894798] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2582.110763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2582.423504] Lustre: 92875:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2589.064650] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2592.878627] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2592.990057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2598.514868] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2604.221838] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2604.543456] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2609.926733] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2610.329977] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2613.732233] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2613.746759] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2613.753609] LustreError: Skipped 1 previous similar message [ 2613.765907] Lustre: Skipped 1 previous similar message [ 2613.772103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2613.799406] Lustre: Skipped 8 previous similar messages [ 2627.039326] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2627.511507] Lustre: server umount lustre-OST0000 complete [ 2632.170796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2638.087880] Lustre: server umount lustre-MDT0000 complete [ 2639.330425] LustreError: 90916:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2639.348161] LustreError: 90916:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [ 2641.788414] LustreError: 94419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786180038 with bad export cookie 4146636619790886266 [ 2641.800602] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2641.806828] LustreError: 94419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2642.163432] Lustre: server umount lustre-MDT0001 complete [ 2650.695192] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:07:26 (1786180046) [ 2658.084590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2662.641673] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2671.496513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2671.871196] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2671.879871] Lustre: Skipped 3 previous similar messages [ 2676.873928] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2685.483461] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2691.944775] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2698.336938] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2705.557539] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2714.195484] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2718.349251] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2718.550826] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2723.949991] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2724.117095] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2731.568970] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2731.944350] Lustre: lustre-OST0001: new disk, initializing [ 2731.952572] Lustre: Skipped 2 previous similar messages [ 2731.961324] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2731.976405] Lustre: Skipped 3 previous similar messages [ 2731.996552] Lustre: 98661:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2740.466905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2740.499488] Lustre: Skipped 2 previous similar messages [ 2740.524171] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2740.774460] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2741.491522] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2753.007201] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2759.340784] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2759.604463] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2764.923764] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2765.192324] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2784.458147] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2810.732787] Lustre: Failing over lustre-MDT0000 [ 2811.132971] Lustre: server umount lustre-MDT0000 complete [ 2812.388742] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2812.395537] LustreError: 95292:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2812.408693] Lustre: Skipped 5 previous similar messages [ 2812.426292] LustreError: 95292:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 6 previous similar messages [ 2827.751627] Lustre: 66690:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786180208/real 1786180208] req@0000000017591b24 x1872944720790400/t0(0) o400->MGC192.168.202.128@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786180224 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2827.782580] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2836.002185] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2836.004556] LDISKFS-fs (dm-0): recovery complete [ 2836.010271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2838.487761] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2840.051668] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2840.056174] Lustre: 95287:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2843.627921] Lustre: 95292:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2843.651316] Lustre: 95292:0:(ldlm_lib.c:2083:extend_recovery_timer()) Skipped 1 previous similar message [ 2843.670496] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2843.774272] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 2843.797965] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2844.119235] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2844.445222] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2844.454139] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 1 previous similar message [ 2844.786492] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2844.796778] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 1 previous similar message [ 2844.809615] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2844.830476] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) Skipped 7 previous similar messages [ 2845.484271] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2845.502631] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 2 previous similar messages [ 2845.832545] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2845.845615] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 2 previous similar messages [ 2846.824657] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2846.843840] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) Skipped 5 previous similar messages [ 2847.808213] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2847.830959] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 6 previous similar messages [ 2848.151108] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2848.159505] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 6 previous similar messages [ 2851.021567] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2851.041210] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) Skipped 12 previous similar messages [ 2852.019270] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2852.034946] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 12 previous similar messages [ 2852.359576] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2852.370108] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 12 previous similar messages [ 2859.048621] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2859.055887] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) Skipped 24 previous similar messages [ 2860.329502] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2860.337903] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 25 previous similar messages [ 2860.655131] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2860.663757] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 25 previous similar messages [ 2875.319373] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2875.332135] Lustre: 101511:0:(ldlm_lib.c:2083:extend_recovery_timer()) Skipped 50 previous similar messages [ 2876.609842] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2876.626984] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 50 previous similar messages [ 2876.959220] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2876.967045] LustreError: 101511:0:(ldlm_lib.c:2696:replay_request_or_update()) Skipped 50 previous similar messages [ 2900.087176] Lustre: 101511:0:(ldlm_lib.c:2392:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2900.097628] Lustre: 101511:0:(ldlm_lib.c:2402:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2900.109668] Lustre: 101511:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 247938c7-d58c-4b48-bb68-faa19e333048@192.168.202.28@tcp [ 2900.123481] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2900.138177] Lustre: 101511:0:(ldlm_lib.c:1912:abort_req_replay_queue()) @@@ aborted: req@0000000071adbdf6 x1872944706949120/t0(8589934773) o101->247938c7-d58c-4b48-bb68-faa19e333048@192.168.202.28@tcp:552/0 lens 592/0 e 0 to 0 dl 1786180307 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2900.164221] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2900.187699] Lustre: lustre-MDT0000: Denying connection for new client 247938c7-d58c-4b48-bb68-faa19e333048 (at 192.168.202.28@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2900.194523] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2900.291793] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2900.301832] Lustre: Skipped 2 previous similar messages [ 2900.305587] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2900.348685] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2900.349909] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2920.427978] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2920.437544] Lustre: Skipped 8 previous similar messages [ 2925.781495] Lustre: server umount lustre-OST0000 complete [ 2930.680870] LustreError: 97116:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2930.704854] LustreError: 97116:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 33 previous similar messages [ 2932.196200] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2937.146913] Lustre: server umount lustre-OST0001 complete [ 2941.409023] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2941.416733] Lustre: Skipped 5 previous similar messages [ 2948.009411] Lustre: server umount lustre-MDT0000 complete [ 2953.101438] LustreError: 95272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786180349 with bad export cookie 4146636619790937478 [ 2953.128104] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2953.802329] Lustre: server umount lustre-MDT0001 complete [ 2964.812850] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:12:39 (1786180359) [ 2974.582995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2974.586977] Lustre: *** cfs_fail_loc=197, val=0*** [ 2990.050238] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2995.169299] LustreError: 103498:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.128@tcp: failed processing log, type 4: rc = -110 [ 3000.309581] Lustre: *** cfs_fail_loc=197, val=0*** [ 3000.313282] Lustre: Skipped 65 previous similar messages [ 3020.901293] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3020.903295] Lustre: Skipped 3 previous similar messages [ 3027.742455] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3035.184112] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3037.221190] Lustre: server umount lustre-OST0000 complete [ 3047.217957] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:14:01 (1786180441) [ 3069.321691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3069.779961] LustreError: 105117:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3069.787972] LustreError: 105117:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 4 previous similar messages [ 3073.887045] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3083.542605] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3088.753542] LustreError: 105148:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3088.768439] LustreError: 105148:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3088.780367] LustreError: 105148:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3089.367676] Lustre: server umount lustre-MDT0000 complete [ 3125.059491] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 3135.817053] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 3175.677124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3175.946871] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3175.994939] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3176.110759] Lustre: lustre-MDT0000: new disk, initializing [ 3176.194937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3181.052708] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3191.736137] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3191.907388] Lustre: 109327:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3192.262235] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3196.970206] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3207.720666] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3216.317776] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3226.683885] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3227.231383] Lustre: 111296:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3233.193918] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3233.466193] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3243.269452] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3248.346584] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3248.560264] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3253.431837] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3253.698981] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3259.374885] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3259.387375] LustreError: Skipped 2 previous similar messages [ 3259.391803] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3259.396382] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3259.396391] Lustre: Skipped 8 previous similar messages [ 3259.418431] Lustre: Skipped 2 previous similar messages [ 3270.111816] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3270.378471] Lustre: server umount lustre-OST0000 complete [ 3281.155412] Lustre: server umount lustre-MDT0000 complete [ 3286.051743] LustreError: 109319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786180682 with bad export cookie 4146636619790948097 [ 3286.063172] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3286.069072] LustreError: 109319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3286.563320] Lustre: server umount lustre-MDT0001 complete [ 3299.931730] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:18:14 (1786180694) [ 3310.738759] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 3314.756686] Key type lgssc unregistered [ 3314.999471] LNet: 113789:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3315.005686] LNetError: 113789:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3315.017473] LNet: Removed LNI 192.168.202.128@tcp [ 3315.843459] Key type .llcrypt unregistered [ 3315.845461] Key type ._llcrypt unregistered [ 3337.680417] Key type ._llcrypt registered [ 3337.686390] Key type .llcrypt registered [ 3337.878359] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 3358.688714] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 3359.804320] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3359.940306] alg: No test for adler32 (adler32-zlib) [ 3361.166901] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 3361.441634] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 3363.159638] Key type lgssc registered [ 3364.415547] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3415.425539] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3415.475702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3416.796280] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3416.859317] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3416.971291] Lustre: lustre-MDT0000: new disk, initializing [ 3417.069632] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3417.100329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3421.439773] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3432.995338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3433.486062] Lustre: lustre-OST0000: new disk, initializing [ 3433.494557] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3433.505990] Lustre: Skipped 1 previous similar message [ 3433.512490] Lustre: 117965:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3433.643837] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3435.659509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3435.680293] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3435.712879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3441.372679] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3453.879718] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3454.100671] Lustre: lustre-OST0001: new disk, initializing [ 3454.103926] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3454.109995] Lustre: 118829:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3454.206209] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3461.331049] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3462.832142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3462.843176] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3462.891641] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3473.379213] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3473.383588] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3473.404663] Lustre: Skipped 1 previous similar message [ 3473.419254] Lustre: Skipped 1 previous similar message [ 3476.918949] Lustre: server umount lustre-MDT0000 complete [ 3484.072813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3489.071402] LustreError: 117154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786180885 with bad export cookie 803488174461270084 [ 3489.082905] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3489.091384] LustreError: 117154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3489.172990] Lustre: server umount lustre-OST0000 complete [ 3493.269600] Lustre: server umount lustre-OST0001 complete [ 3502.952937] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 05:21:37 (1786180897) [ 3528.957878] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 05:22:03 (1786180923) [ 3541.160212] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 3567.253908] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:22:41 (1786180961) [ 3616.868452] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 3630.858550] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 3675.626246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3675.895234] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3675.953788] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3676.038532] Lustre: lustre-MDT0000: new disk, initializing [ 3676.143640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3676.167933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3681.471716] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3693.846527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3694.037774] Lustre: 126587:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3694.119804] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3694.133772] Lustre: Skipped 1 previous similar message [ 3694.290748] Lustre: lustre-MDT0001: new disk, initializing [ 3694.424761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3694.514521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3694.531666] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3700.121123] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3710.886767] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3718.302610] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3726.808317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3727.200654] Lustre: lustre-OST0000: new disk, initializing [ 3727.203191] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3727.211429] Lustre: 128560:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3727.355206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3734.951716] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3735.676162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3735.702772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3735.857434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3746.245471] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3753.068904] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3753.630980] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3759.835577] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3759.947558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3766.754931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3766.762293] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3766.764910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3766.771776] Lustre: Skipped 1 previous similar message [ 3768.571572] Lustre: server umount lustre-OST0000 complete [ 3776.486746] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3776.489169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3776.495474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3776.498669] Lustre: Skipped 1 previous similar message [ 3782.112898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3782.116992] Lustre: Skipped 2 previous similar messages [ 3787.232584] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3787.240083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3787.260892] Lustre: Skipped 1 previous similar message [ 3787.529978] Lustre: server umount lustre-MDT0000 complete [ 3791.865491] LustreError: 126580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786181188 with bad export cookie 803488174461271316 [ 3791.870264] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3791.889496] LustreError: 126580:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3792.447645] Lustre: server umount lustre-MDT0001 complete [ 3802.188797] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:26:37 (1786181197) [ 3810.482548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3811.316617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3816.111277] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3816.420462] LustreError: 130975:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3816.451224] LustreError: 130975:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 3821.538607] LustreError: 130974:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3825.763196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3830.801233] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3840.271641] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3847.384458] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3854.646239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3855.188349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3855.201458] Lustre: Skipped 1 previous similar message [ 3863.453256] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 3872.483755] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3878.059626] Lustre: *** cfs_fail_loc=159, val=0*** [ 3894.346147] Lustre: lustre-MDT0000: Client 22974d82-b60e-4ada-a10f-ce94c31d7224 (at 192.168.202.28@tcp) reconnecting [ 3896.485027] Lustre: *** cfs_fail_loc=159, val=0*** [ 3896.486832] Lustre: Skipped 5 previous similar messages [ 3912.781854] Lustre: lustre-MDT0000: Client 22974d82-b60e-4ada-a10f-ce94c31d7224 (at 192.168.202.28@tcp) reconnecting [ 3921.894822] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3921.896790] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3921.914308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3921.924298] Lustre: Skipped 2 previous similar messages [ 3921.944759] Lustre: Skipped 1 previous similar message [ 3925.407390] Lustre: server umount lustre-OST0000 complete [ 3932.129671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3932.141641] Lustre: Skipped 1 previous similar message [ 3936.028536] Lustre: server umount lustre-MDT0000 complete [ 3937.252273] LustreError: 131703:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3937.279866] LustreError: 131703:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 3940.815208] LustreError: 130954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786181337 with bad export cookie 803488174461272198 [ 3940.834121] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3940.835857] LustreError: 130954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3941.597642] Lustre: server umount lustre-MDT0001 complete [ 3955.236435] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 3959.191955] Key type lgssc unregistered [ 3959.660524] LNet: 134753:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3959.678468] LNetError: 134753:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3960.753502] LNet: Removed LNI 192.168.202.128@tcp [ 3961.672831] Key type .llcrypt unregistered [ 3961.674517] Key type ._llcrypt unregistered [ 3988.416539] Key type ._llcrypt registered [ 3988.423184] Key type .llcrypt registered [ 3988.602922] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 3990.439225] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3990.499185] alg: No test for adler32 (adler32-zlib) [ 3991.585336] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 3991.923266] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 3993.671604] Key type lgssc registered [ 3995.174119] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4004.149433] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4004.217715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4005.875575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4010.804832] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4016.099267] LustreError: 136169:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4016.147728] LustreError: 136169:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 4021.024666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4021.292828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4026.489431] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4035.776158] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4044.072311] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4052.774116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4053.104060] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4060.383480] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4069.255930] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4075.653871] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4088.795045] Lustre: *** cfs_fail_loc=159, val=0*** [ 4105.278354] Lustre: lustre-MDT0000: Client 584450c2-d3f5-4b61-9b26-e70bfce62bc7 (at 192.168.202.28@tcp) reconnecting [ 4107.874844] Lustre: *** cfs_fail_loc=159, val=0*** [ 4107.887282] Lustre: Skipped 1 previous similar message [ 4124.208504] Lustre: lustre-MDT0001: Client 584450c2-d3f5-4b61-9b26-e70bfce62bc7 (at 192.168.202.28@tcp) reconnecting [ 4125.923333] Lustre: *** cfs_fail_loc=159, val=0*** [ 4125.927384] Lustre: Skipped 2 previous similar messages [ 4141.672391] Lustre: lustre-MDT0001: Client 584450c2-d3f5-4b61-9b26-e70bfce62bc7 (at 192.168.202.28@tcp) reconnecting [ 4145.978766] Lustre: 139236:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4149.501489] Lustre: *** cfs_fail_loc=159, val=0*** [ 4149.503252] Lustre: Skipped 4 previous similar messages [ 4165.721943] Lustre: lustre-MDT0000: Client 4b9d21b0-6bfa-46f0-83c9-51e24210a908 (at 192.168.202.28@tcp) reconnecting [ 4168.296407] Lustre: *** cfs_fail_loc=159, val=0*** [ 4168.313809] Lustre: Skipped 13 previous similar messages [ 4184.609664] Lustre: lustre-MDT0000: Client 4b9d21b0-6bfa-46f0-83c9-51e24210a908 (at 192.168.202.28@tcp) reconnecting [ 4186.421407] Lustre: 139525:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4205.024553] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4205.043426] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4205.067507] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4206.900071] Lustre: server umount lustre-OST0000 complete [ 4214.244969] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4214.257928] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4214.269056] Lustre: Skipped 1 previous similar message [ 4214.292205] Lustre: Skipped 1 previous similar message [ 4219.361060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4221.528368] Lustre: server umount lustre-MDT0000 complete [ 4224.483045] LustreError: 139135:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4224.513524] LustreError: 139135:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 4225.774782] LustreError: 136150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786181622 with bad export cookie 8778543346253516891 [ 4225.798679] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4226.189806] Lustre: server umount lustre-MDT0001 complete [ 4238.336792] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 4241.630277] Key type lgssc unregistered [ 4242.108884] LNet: 140605:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4242.115139] LNetError: 140605:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4242.128326] LNet: Removed LNI 192.168.202.128@tcp [ 4243.328380] Key type .llcrypt unregistered [ 4243.331114] Key type ._llcrypt unregistered [ 4271.069506] Key type ._llcrypt registered [ 4271.086239] Key type .llcrypt registered [ 4271.244283] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 4272.767088] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4272.820150] alg: No test for adler32 (adler32-zlib) [ 4273.917828] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 4274.279115] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 4275.952447] Key type lgssc registered [ 4277.357981] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4287.754404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4287.799149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4289.615613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4297.264698] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4299.753613] LustreError: 142020:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4299.795821] LustreError: 142020:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 4300.770181] LustreError: 142019:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4305.899615] LustreError: 142020:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4309.901396] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4310.470047] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4315.740577] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4325.171182] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4331.845741] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4340.247994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4340.704414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4349.024823] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4355.065270] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4357.849428] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4361.223943] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4368.064513] Lustre: 144604:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4371.526281] Lustre: 144652:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4373.292897] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4374.347578] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4378.223496] Lustre: 144799:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4379.470080] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4380.549196] Lustre: 144897:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4385.760480] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4385.769261] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4385.783689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4386.791930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4386.796381] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4386.800223] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4386.813632] Lustre: Skipped 1 previous similar message [ 4391.953083] Lustre: server umount lustre-OST0000 complete [ 4397.041959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4397.063595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4397.081605] Lustre: Skipped 1 previous similar message [ 4402.156304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4402.171670] Lustre: Skipped 1 previous similar message [ 4402.915487] Lustre: server umount lustre-MDT0000 complete [ 4406.903081] LustreError: 142000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786181803 with bad export cookie 3117504728048457202 [ 4406.906138] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4406.910136] LustreError: 142000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4407.148809] Lustre: server umount lustre-MDT0001 complete [ 4423.732910] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 4427.669209] Key type lgssc unregistered [ 4427.988257] LNet: 145963:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4428.003481] LNetError: 145963:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4429.044287] LNet: Removed LNI 192.168.202.128@tcp [ 4430.552409] Key type .llcrypt unregistered [ 4430.554880] Key type ._llcrypt unregistered [ 4462.288450] Key type ._llcrypt registered [ 4462.291547] Key type .llcrypt registered [ 4462.412879] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 4463.604366] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4463.644238] alg: No test for adler32 (adler32-zlib) [ 4465.123224] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 4465.342910] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 4467.031167] Key type lgssc registered [ 4468.330738] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4477.703463] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4477.719237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4479.300818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4484.929739] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4488.679188] LustreError: 147383:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4488.708632] LustreError: 147383:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 4493.795777] LustreError: 147384:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4495.698452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4496.199715] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4500.521202] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4509.014484] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4516.171647] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4524.192783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4524.594593] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4525.647252] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4531.650314] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4541.071675] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4541.980310] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4548.940934] Lustre: *** cfs_fail_loc=159, val=0*** [ 4564.527961] Lustre: lustre-MDT0000: Client 44cbed65-dc5e-4007-acec-7f83153ce959 (at 192.168.202.28@tcp) reconnecting [ 4571.617266] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4571.626148] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4571.659743] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4572.133114] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4572.146590] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4572.175406] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4572.180453] Lustre: Skipped 1 previous similar message [ 4577.026754] Lustre: server umount lustre-OST0000 complete [ 4582.368536] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4582.372225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4582.392499] Lustre: Skipped 1 previous similar message [ 4582.416764] Lustre: Skipped 1 previous similar message [ 4587.491309] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4587.507923] Lustre: Skipped 1 previous similar message [ 4587.761594] Lustre: server umount lustre-MDT0000 complete [ 4591.586141] LustreError: 148420:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4591.604283] LustreError: 148420:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 4591.976183] LustreError: 147365:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786181988 with bad export cookie 5709838017411893858 [ 4591.987704] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4592.309847] Lustre: server umount lustre-MDT0001 complete [ 4605.987691] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 4609.454966] Key type lgssc unregistered [ 4609.799856] LNet: 151047:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4609.802831] LNetError: 151047:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4609.819282] LNet: Removed LNI 192.168.202.128@tcp [ 4611.206674] Key type .llcrypt unregistered [ 4611.209775] Key type ._llcrypt unregistered [ 4640.990785] Key type ._llcrypt registered [ 4640.993343] Key type .llcrypt registered [ 4641.096783] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 4643.029555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4643.085022] alg: No test for adler32 (adler32-zlib) [ 4644.370882] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 4644.770758] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 4646.487957] Key type lgssc registered [ 4647.764164] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4656.381432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4656.399167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4658.046810] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4663.115021] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4668.390229] LustreError: 152463:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4668.402978] LustreError: 152463:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 4673.041895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4673.393416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4678.884908] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4688.097516] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4695.311949] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4702.957760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4703.215645] LustreError: 154289:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4703.353138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4708.891052] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4710.317439] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4718.074347] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4718.733727] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4738.204424] Lustre: lustre-OST0000: evicting 11b32d12-7de8-4c29-b045-75b7dd56b4de (at 192.168.202.28@tcp) by administrative request [ 4738.217448] Lustre: Skipped 1 previous similar message [ 4769.250989] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4769.280546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4769.296850] Lustre: Skipped 1 previous similar message [ 4774.383269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4774.403071] Lustre: Skipped 1 previous similar message [ 4775.812351] Lustre: server umount lustre-OST0000 complete [ 4784.609520] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4784.611331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4784.630821] Lustre: Skipped 2 previous similar messages [ 4784.652636] Lustre: Skipped 1 previous similar message [ 4787.744612] Lustre: server umount lustre-MDT0000 complete [ 4789.728196] LustreError: 152458:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4789.738529] LustreError: 152458:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 4791.509210] LustreError: 152444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786182187 with bad export cookie 11525354611740522031 [ 4791.522994] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4791.523876] LustreError: 152444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4791.968101] Lustre: server umount lustre-MDT0001 complete [ 4806.799416] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 4809.647481] Key type lgssc unregistered [ 4809.890671] LNet: 156538:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4809.893147] LNetError: 156538:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4809.908098] LNet: Removed LNI 192.168.202.128@tcp [ 4810.865702] Key type .llcrypt unregistered [ 4810.867544] Key type ._llcrypt unregistered [ 4906.976663] Key type ._llcrypt registered [ 4906.978867] Key type .llcrypt registered [ 4907.109922] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 4908.499211] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4908.534830] alg: No test for adler32 (adler32-zlib) [ 4909.666228] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 4910.009893] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 4911.775183] Key type lgssc registered [ 4913.282800] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4922.354652] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4922.371317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4923.892064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4928.920757] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4934.114678] LustreError: 159033:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4934.132377] LustreError: 159033:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 4939.106152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4939.232925] LustreError: 159034:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4939.762087] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4945.016620] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4953.449746] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4960.575432] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4968.598985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4969.171836] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4970.251995] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4977.217773] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 4984.633173] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4986.528749] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5000.162384] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5000.171183] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5000.179192] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5000.203115] Lustre: Skipped 1 previous similar message [ 5002.790132] Lustre: server umount lustre-OST0000 complete [ 5010.404765] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5010.416686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5010.422399] Lustre: Skipped 1 previous similar message [ 5010.449737] Lustre: Skipped 2 previous similar messages [ 5013.868795] Lustre: server umount lustre-MDT0000 complete [ 5015.529960] LustreError: 159033:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5015.564702] LustreError: 159033:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 5018.051592] LustreError: 159015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786182414 with bad export cookie 16514751411459810861 [ 5018.061628] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5018.066280] LustreError: 159015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5018.353100] Lustre: server umount lustre-MDT0001 complete [ 5032.744433] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 5036.252712] Key type lgssc unregistered [ 5036.539650] LNet: 162579:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5036.556165] LNetError: 162579:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5036.598834] LNet: Removed LNI 192.168.202.128@tcp [ 5037.769601] Key type .llcrypt unregistered [ 5037.771390] Key type ._llcrypt unregistered [ 5068.915885] Key type ._llcrypt registered [ 5068.918542] Key type .llcrypt registered [ 5069.061275] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 5070.817594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5070.873027] alg: No test for adler32 (adler32-zlib) [ 5071.901398] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 5072.205942] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 5073.919390] Key type lgssc registered [ 5075.132181] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5093.593536] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:48:08 (1786182488) [ 5114.059547] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 5125.900941] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 5171.604192] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5171.620151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5172.974994] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5173.044941] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5173.172280] Lustre: lustre-MDT0000: new disk, initializing [ 5173.303459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5173.336069] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5178.767375] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 5190.440806] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5190.679671] Lustre: 166849:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5190.725292] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5190.728271] Lustre: Skipped 1 previous similar message [ 5190.828242] Lustre: lustre-MDT0001: new disk, initializing [ 5190.958618] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5191.003841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5191.031225] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5195.678264] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 5206.213377] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5214.039608] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5221.907671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5222.180797] Lustre: lustre-OST0000: new disk, initializing [ 5222.188740] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5222.194235] Lustre: 168819:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5222.322591] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5228.643712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5228.664507] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5228.772470] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5229.784798] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 5240.558609] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5249.506209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5249.509396] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5249.530682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5249.557389] Lustre: Skipped 1 previous similar message [ 5254.635350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5254.643446] Lustre: Skipped 1 previous similar message [ 5255.664453] Lustre: server umount lustre-OST0000 complete [ 5262.822765] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5262.839593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5262.861799] Lustre: Skipped 1 previous similar message [ 5262.871191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5264.869811] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5264.883810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5264.893664] Lustre: Skipped 1 previous similar message [ 5266.608933] Lustre: server umount lustre-MDT0000 complete [ 5271.248750] LustreError: 166840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786182667 with bad export cookie 9934743385408196202 [ 5271.255205] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5271.257729] LustreError: 166840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5271.945738] Lustre: server umount lustre-MDT0001 complete [ 5284.521408] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 5287.178235] Key type lgssc unregistered [ 5287.455637] LNet: 170730:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5287.460397] LNetError: 170730:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5288.489891] LNet: Removed LNI 192.168.202.128@tcp [ 5289.392705] Key type .llcrypt unregistered [ 5289.395164] Key type ._llcrypt unregistered [ 5317.060710] Key type ._llcrypt registered [ 5317.062796] Key type .llcrypt registered [ 5317.185707] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 5318.405179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5318.469954] alg: No test for adler32 (adler32-zlib) [ 5319.679676] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 5319.952278] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 5321.687266] Key type lgssc registered [ 5322.934954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5331.962873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5331.975854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5333.386662] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5338.190199] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 5342.700131] LustreError: 172148:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5342.728902] LustreError: 172148:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 5347.576454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5347.856042] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5352.511060] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 5361.114655] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5368.033503] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5376.214485] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5376.664722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5377.724371] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5385.694676] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 5395.859861] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5403.410607] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5403.438212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5403.455337] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5403.468564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.495304] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5404.528982] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5404.541300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5404.550781] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5404.555095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.563044] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5405.477657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.283980] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5406.290520] Lustre: Skipped 1 previous similar message [ 5406.304736] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5406.315612] Lustre: Skipped 1 previous similar message [ 5406.319037] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5406.324186] Lustre: Skipped 1 previous similar message [ 5406.328359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.338670] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5406.345815] Lustre: Skipped 1 previous similar message [ 5407.405235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.323989] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5408.327147] Lustre: Skipped 1 previous similar message [ 5408.342864] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5408.349933] Lustre: Skipped 1 previous similar message [ 5408.353726] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5408.357186] Lustre: Skipped 1 previous similar message [ 5408.362771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.384258] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5408.388942] Lustre: Skipped 1 previous similar message [ 5409.423598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.466253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.682802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.039633] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5413.053792] Lustre: Skipped 3 previous similar messages [ 5413.073819] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5413.087183] Lustre: Skipped 3 previous similar messages [ 5413.091201] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5413.094224] Lustre: Skipped 3 previous similar messages [ 5413.098834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.116863] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5413.134976] Lustre: Skipped 3 previous similar messages [ 5414.150925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.181037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.181802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.093216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.037261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.021314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.070867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.057155] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5421.061844] Lustre: Skipped 7 previous similar messages [ 5421.088452] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5421.102484] Lustre: Skipped 7 previous similar messages [ 5421.112558] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5421.122567] Lustre: Skipped 7 previous similar messages [ 5421.127929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.134960] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5421.143453] Lustre: Skipped 7 previous similar messages [ 5421.945469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.955869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.039792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.794730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.839611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.780554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.863672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.834738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.967985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.904194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.143681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.284901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.691625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.778660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.910764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.038897] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5438.049477] Lustre: Skipped 15 previous similar messages [ 5438.087537] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5438.099226] Lustre: Skipped 15 previous similar messages [ 5438.105320] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5438.114674] Lustre: Skipped 15 previous similar messages [ 5438.119235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.127837] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5438.133526] Lustre: Skipped 15 previous similar messages [ 5439.184954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.063796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.040145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.086275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.104183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.196986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.142640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.128421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.311535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.387475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.170940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.123277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.177098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.215358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.310828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.272639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.289475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.151509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.035210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.962138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.874123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.829550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.672253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.604508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.559682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.395854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.377249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.478243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.426924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.478386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.424616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.333607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.136582] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5470.143617] Lustre: Skipped 32 previous similar messages [ 5470.155729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5470.165420] Lustre: Skipped 32 previous similar messages [ 5470.174945] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5470.183500] Lustre: Skipped 32 previous similar messages [ 5470.195086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.204525] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5470.209153] Lustre: Skipped 32 previous similar messages [ 5471.156488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.033223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.973553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.011524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.128197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.175539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.298199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.441958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.605612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.494325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.612605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.528198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.734354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.839591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.762927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.880331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.879635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.949288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.998576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.931107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.945081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.122518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.280512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.364129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.582113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.614858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.586211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.719337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.925681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.920976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.976408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.037238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.975112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.062518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.019331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.009833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.124309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.937072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.865968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.136655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.139538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.243074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.146682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.157337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.152566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.255203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.286092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.304609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.298402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.231286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.069885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.072084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.990477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.955708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.827219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.802200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.783468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.840658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.829986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.827090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.863848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.073719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.063671] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5535.074639] Lustre: Skipped 62 previous similar messages [ 5535.089763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5535.109040] Lustre: Skipped 62 previous similar messages [ 5535.115968] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5535.126141] Lustre: Skipped 62 previous similar messages [ 5535.130111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.145907] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5535.152081] Lustre: Skipped 62 previous similar messages [ 5536.441943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.750887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.655438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.870026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.896369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.853015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.698128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.537463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.621973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.712072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.840640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.778842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.877346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.076577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.494212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.583345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.646867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.806665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.943777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.343509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.695772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.128156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.154211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.066312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.303403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.318400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.394032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.661859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.779825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.887626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.995842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.196960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.376058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.349108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.478903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.544318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.523602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.622737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.454528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.440609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.387882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.256640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.953080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.040823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.012457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.084645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.124390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.160479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.353434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.560929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.668808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.561412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.518329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.365555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.463874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.429274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.678883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.732856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.673755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.647080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.810438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.945243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.035061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.982634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.103919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.142934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.151685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.182841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.140373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.063951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.990558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.011524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.767676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.615930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.479883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.703815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.750972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.615127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.602928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.522910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.470848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.520044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.466693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.475914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.336775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.307785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.225637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.441058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.359611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.402535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.660982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.790084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.798564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.144675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.084136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.181756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.222758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.119953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.062162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.880330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.054876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.020896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.041047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.214994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.121953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.215581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.167432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.053946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.165570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.329622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.365820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.378349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.558213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.581797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.475229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.634512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.810725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.652426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.428645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.475928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.664433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.469855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.112765] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5663.118843] Lustre: Skipped 122 previous similar messages [ 5663.131537] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5663.138525] Lustre: Skipped 122 previous similar messages [ 5663.141429] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5663.143590] Lustre: Skipped 122 previous similar messages [ 5663.145181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.149636] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5663.153752] Lustre: Skipped 122 previous similar messages [ 5664.019565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.956633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.680032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.697301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.685734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.954512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.926797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.020413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.042029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.007296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.792481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.694266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.470620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.808404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.796607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.884608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.981186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.072653] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5681.084607] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5681.093876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.015993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.049216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.893179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.671820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.515724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.410592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.198162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.189988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.126048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.367905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.260725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.338868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.342877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.619240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.767326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.727710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.805286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.758903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.826967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.848379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.767964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.653642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.538505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.498880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.416591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.441639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.339530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.164991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.120640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.074842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.048101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.951792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.794556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.662692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.676222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.679933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.572814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.416276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.605542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.621896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.789179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.029824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.428058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.954661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.909256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.086244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.994726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.047196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.644656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.639585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.373123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.279630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.047349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.847740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.868890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.168095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.179822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.027500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.879649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.920034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.490064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.191781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.389227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.183763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.245458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.179362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.242697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.292830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.048314] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5748.051552] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5748.054841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.620467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.644603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.797414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.785643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.817476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.839561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.972648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.195670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.278401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.319128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.577732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.583026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.789875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.906336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.000540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.934203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.108615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.029203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.002477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.789549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.686698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.656324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.699685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.911642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.997074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.118866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.333654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.596267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.673129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.655353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.711540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.667914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.783346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.880025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.793766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.001061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.053574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.211276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.133394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.278448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.271115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.397772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.528685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.526926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.773141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.860538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.148164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.006956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.238670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.114409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.566919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.725035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.021100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.335261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.589378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.676836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.023442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.403519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.346441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.391479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.541693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.419895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.636718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.892813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.657571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.677083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.670171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.740660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.035908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.094007] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.139888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.867757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.640102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.426536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.418976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.462900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.340726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.329325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.186310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.238173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.120924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.090190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.181955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.366505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.322512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.222299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.181740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.182796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.027402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.170388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.056748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.092981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.149445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.031101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.927964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.986154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.937929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.932250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.981876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.258468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.252442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.241844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.216392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.284904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.338691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.322853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.160955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.050295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.281134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.195453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.162691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.690860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.021527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.390446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.892578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.232259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.498250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.475548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.607175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.079795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.771083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.607874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.375715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.436935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.815747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.170932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.777637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.956283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.011360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.128293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.078264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.050472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.165833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.246734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.295453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.258336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.110665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.062757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.997789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.917147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.891432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.708443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.699845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.440527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.318336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.114302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.987315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.027320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.056905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.232650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.243281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.381279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.663053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.872746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.934969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.834186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.870230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.992109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.800046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.991853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.187102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.541648] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5919.546029] Lustre: Skipped 248 previous similar messages [ 5919.568685] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5919.578820] Lustre: Skipped 248 previous similar messages [ 5919.582333] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5919.585021] Lustre: Skipped 248 previous similar messages [ 5919.589935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.621333] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5919.628477] Lustre: Skipped 248 previous similar messages [ 5920.952247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.202122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.249499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.228707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.567561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.695241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.009669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.150830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.305812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.470696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.323926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.511526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.812718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.817867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.140541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.052920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.137694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.035351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.898099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.877704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.890112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.518821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.462286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.537480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.728765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.885828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.048660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.266085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.669542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.960717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.445336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.728932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.972756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.856121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.052488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.266271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.702776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.744834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.826909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.723490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.731656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.514206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.511439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.474356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.502412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.410852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.176286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.978351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.958448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.877571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.820301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.716473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.586247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.463327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.548661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.431714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.443569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.355687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.580265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.513948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.803307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.909083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.896784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.989246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.991741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.100845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.014718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.033817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.031186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.187699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.298544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.328323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.252052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.580776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.566406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.528288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.464070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.663822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.626655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.844114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.048864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.178513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.175253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.345919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.499940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.515247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.544896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.582329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.665198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.711260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.153170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.304519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.262544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.454741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.411937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.127346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.011771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.854601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.075562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.237759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.218470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.329510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.374576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.751505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.934555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.457319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.992688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.557140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.977033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.356375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.741044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.976469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.049614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.009032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.242791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.439894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.754973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.560724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.481304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.582494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.514498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.532775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.512794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.327106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.119668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.068770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.115429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.215907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.161411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.073311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.753094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.586460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.371250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.424841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.267778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.035654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.143257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.323826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.472025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.668888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.998465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.932391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.957128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.779538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.914698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.768901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.769298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.599274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.707871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.697596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.654746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.768886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.717052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.773875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.902848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.018733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.060879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.052804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.148986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.184059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.110716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.180667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.171737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.151456] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6094.159919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.161817] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6094.976692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.225621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.235399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.390316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.519135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.699476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.931608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.939787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.902275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.869393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.860949] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6105.871815] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6105.872990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.604518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.404952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.292652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.537666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.584853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.617254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.691595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.943102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.902868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.759864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.716732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.313842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.773462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.180237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.944684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.007879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.288342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.816473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.553737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.158421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.304356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.459684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.449192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.534704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.382811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.302575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.449537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.843784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.809344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.926610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.831526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.875660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.831093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.146686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.118193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.157952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.206711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.232995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.367333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.358118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.294468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.307793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.367312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.550089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.707121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.892532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.369972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.479819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.529185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.852953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.423793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.037348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.619084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.730446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.084868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.139818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.682129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.918646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.028392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.210620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.258666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.673525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.872991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.143795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.085252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.920628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.906917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.837483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.200250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.300581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.276659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.166420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.071142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.400226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.534762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.981344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.443517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.376482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.423411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.584806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.561369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.347166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.238739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.081939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.380961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.400178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.630465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.545533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.514854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.420382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.368980] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6208.378914] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6208.390570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.303357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.127189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.066463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.061770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.008819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.162250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.261529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.330492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.464551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.515638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.511078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.442191] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6220.453909] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6220.470633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.331561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.310809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.143918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.208469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.950572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.901284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.883185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.844164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.638925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.703657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.822207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.832527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.009630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.985835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.135042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.059442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.020914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.953393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.946464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.797550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.960793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.256266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.332921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.202587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.216687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.288784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.229526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.061233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.881929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.822705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.952462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.867780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.791948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.799816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.905218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.769496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.749338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.022589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.010566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.947530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.604507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.468653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.256787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.090070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.210779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.246845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.189177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.257940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.358295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.273162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.223868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.152878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.380875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.588394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.714858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.903088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.692302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.590540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.389736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.380775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.451857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.482321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.534138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.503365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.771133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.851524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.789655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.027305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.353174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.426494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.698460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.663572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.735292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.746894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.012635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.983135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.942576] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6297.957464] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6297.965260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.660646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.605753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.321888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.220876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.030448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.908517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.828164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.809971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.771819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.727102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.927751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.975645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.847937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.796518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.942351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.855730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.891165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.807304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.881252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.977438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.948912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.911468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.915892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.032525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.921864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.831177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.612466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.480477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.260347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.302352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.260520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.125902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.100937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.190862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.156446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.249685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.157193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.082271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.137417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.977995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.837735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.697820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.526495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.674110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.671614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.587714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.574614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.720905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.684091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.755836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.667346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.692721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.538380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.623378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.518634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.411434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.340808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.386451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.321026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.329588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.286250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.177519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.075842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.196966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.129230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.075933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.163967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.085900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.196834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.490708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.453951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.722719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.638839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.595759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.808937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.606842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.395890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.222942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.334107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.548509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.898311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.165128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.536396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.723352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.944285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.033192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.144450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.330459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.262400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.249522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.260225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.340810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.358898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.525854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.593822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.494560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.698840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.790805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.912483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.890598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.930574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.173964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.515318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.403891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.582612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.644811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.864974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.989223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.063892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.176492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.210031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.092254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.129318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.144974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.297973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.279652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.404353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.348285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.436475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.467993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.529834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.442458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.402546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.486898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.267342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.214752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.194950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.259645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.181335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.165160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.113150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.256438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.199741] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6432.204604] Lustre: Skipped 487 previous similar messages [ 6432.212879] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6432.220469] Lustre: Skipped 487 previous similar messages [ 6432.224055] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6432.227937] Lustre: Skipped 487 previous similar messages [ 6432.232848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.241787] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6432.251181] Lustre: Skipped 487 previous similar messages [ 6433.137818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.173038] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6434.186690] LustreError: 172168:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6434.208211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.355320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.330654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.498982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.711810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.955655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.007241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.235901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.604820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.044059] LustreError: 174039:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6886.047128] LustreError: 172168:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7562.725743] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7562.738798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7562.755068] Lustre: Skipped 10 previous similar messages [ 7562.763798] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7566.033245] Lustre: server umount lustre-OST0000 complete [ 7573.481797] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7573.497342] Lustre: Skipped 3 previous similar messages [ 7576.753481] Lustre: server umount lustre-MDT0000 complete [ 7578.598744] LustreError: 173377:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7578.633736] LustreError: 173377:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 7581.135034] LustreError: 172130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786184977 with bad export cookie 523037220805674451 [ 7581.146861] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7581.736222] Lustre: server umount lustre-MDT0001 complete [ 7593.610953] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 7596.543433] Key type lgssc unregistered [ 7596.878047] LNet: 227440:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7596.888931] LNetError: 227440:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7596.899317] LNet: Removed LNI 192.168.202.128@tcp [ 7597.751246] Key type .llcrypt unregistered [ 7597.753527] Key type ._llcrypt unregistered [ 7622.808651] Key type ._llcrypt registered [ 7622.810340] Key type .llcrypt registered [ 7622.916689] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 7623.906369] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7623.941980] alg: No test for adler32 (adler32-zlib) [ 7625.260752] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 7625.537334] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 7627.263289] Key type lgssc registered [ 7628.691702] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7638.243204] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7638.286254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7639.919383] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7644.704587] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7650.274311] LustreError: 228856:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7650.298606] LustreError: 228856:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 7652.973219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7653.364592] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7658.307690] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7666.863816] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7672.498430] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7678.782330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7679.159775] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7685.230544] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7685.297613] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7692.846513] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7701.444880] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7703.335812] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7703.347331] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7703.356726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7703.371613] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7703.372164] LustreError: 228876:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7705.611319] Lustre: lustre-OST0000: Not available for connect from 192.168.202.28@tcp (stopping) [ 7708.130488] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7708.147966] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7708.160711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7710.181722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7710.191754] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7710.206973] Lustre: Skipped 1 previous similar message [ 7711.769406] Lustre: server umount lustre-OST0000 complete [ 7723.431305] LustreError: 228876:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7746.383669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7746.740260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7749.819304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7749.842238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7752.196312] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7758.577552] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7760.110286] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7760.129347] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7763.076626] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 7768.004854] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9648d0573800.ost_server_uuid 50 [ 7769.251159] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9648d0573800.ost_server_uuid in FULL state after 0 sec [ 7774.691834] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7774.701545] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7774.718194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7774.720890] Lustre: Skipped 1 previous similar message [ 7775.712366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7779.193704] Lustre: server umount lustre-OST0000 complete [ 7785.960907] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7785.967559] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7785.983063] Lustre: Skipped 2 previous similar messages [ 7786.005133] Lustre: Skipped 3 previous similar messages [ 7789.245381] Lustre: server umount lustre-MDT0000 complete [ 7791.083294] LustreError: 228856:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7791.098443] LustreError: 228856:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 7793.639800] LustreError: 228837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786185190 with bad export cookie 17323753253258332698 [ 7793.648158] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7793.675956] LustreError: 228837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7794.211256] Lustre: server umount lustre-MDT0001 complete [ 7809.408397] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 7812.912353] Key type lgssc unregistered [ 7813.405918] LNet: 234069:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7813.430383] LNetError: 234069:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7813.466687] LNet: Removed LNI 192.168.202.128@tcp [ 7814.934981] Key type .llcrypt unregistered [ 7814.936866] Key type ._llcrypt unregistered [ 7843.354613] Key type ._llcrypt registered [ 7843.356405] Key type .llcrypt registered [ 7843.443092] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 7844.331429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7844.381035] alg: No test for adler32 (adler32-zlib) [ 7845.362676] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 7845.592552] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 7847.319874] Key type lgssc registered [ 7848.390679] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7857.393878] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7857.405276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7858.829549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7863.747612] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7869.415432] LustreError: 235792:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7869.455067] LustreError: 235792:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 7873.391277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7873.791124] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7880.441779] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7884.452923] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7893.270967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7893.652352] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7900.817718] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7903.954810] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7910.345231] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7910.547514] Lustre: lustre-OST0001: new disk, initializing [ 7910.551452] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7910.560513] Lustre: 238101:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7910.670957] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7917.237351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7917.253827] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7917.395855] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7917.807839] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 7928.755520] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7941.228141] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7958.502798] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7958.519267] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7958.530890] Lustre: Skipped 2 previous similar messages [ 7958.548921] Lustre: Skipped 3 previous similar messages [ 7963.621161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7963.634040] Lustre: Skipped 3 previous similar messages [ 7964.220480] Lustre: server umount lustre-MDT0000 complete [ 7968.745967] LustreError: 235792:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7968.759493] LustreError: 235792:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [ 7968.779469] LustreError: 235771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786185365 with bad export cookie 6076516885390882836 [ 7968.786774] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7968.788613] LustreError: 235771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7969.416684] Lustre: server umount lustre-MDT0001 complete [ 7984.298630] Lustre: server umount lustre-OST0000 complete [ 7997.828415] Lustre: server umount lustre-OST0001 complete [ 8020.332620] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 8032.566889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8032.909180] LustreError: 241211:0:(ldlm_lib.c:1189:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8033.040487] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8038.362753] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8038.370882] LustreError: 241212:0:(ldlm_lib.c:1189:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8038.401089] LustreError: 241212:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 8043.497671] LustreError: 241211:0:(ldlm_lib.c:1189:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8048.408692] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8048.623447] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (not set up) [ 8048.900215] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8054.264053] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8059.927335] Lustre: 242355:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8067.408348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8067.969653] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8074.106900] LustreError: 242709:0:(ldlm_lib.c:1189:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8074.125070] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8074.523607] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8078.281887] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8084.340782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8089.612523] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8092.298267] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8100.017978] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8103.907547] Lustre: 244224:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8115.180467] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8115.188739] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8115.192128] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8115.209345] Lustre: Skipped 4 previous similar messages [ 8115.215635] Lustre: Skipped 2 previous similar messages [ 8120.302530] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8120.311676] Lustre: Skipped 3 previous similar messages [ 8120.967069] Lustre: server umount mylustre-MDT0000 complete [ 8125.040555] LustreError: 241186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786185521 with bad export cookie 6076516885390885692 [ 8125.058780] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8125.060964] LustreError: 241186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8125.519539] Lustre: server umount mylustre-MDT0001 complete [ 8140.505137] Lustre: server umount mylustre-OST0000 complete [ 8140.769439] Lustre: 235055:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786185521/real 1786185521] req@ffff999524378000 x1872950976481280/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786185537 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8140.805477] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8144.633367] Lustre: server umount mylustre-OST0001 complete [ 8166.509511] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 8176.887883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8177.428367] LustreError: 246532:0:(ldlm_lib.c:1189:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8177.457309] LustreError: 246532:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 4 previous similar messages [ 8177.543037] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8177.549804] Lustre: Skipped 1 previous similar message [ 8182.577712] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8192.128210] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8197.728772] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8200.760233] Lustre: 247675:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8208.064556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8215.459872] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8217.680885] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8217.802311] LustreError: 248029:0:(ldlm_lib.c:1189:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8217.839012] LustreError: 248029:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [ 8221.809994] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8223.829981] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8229.357552] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 8229.742867] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8236.967612] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8240.372910] Lustre: 249546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8249.312092] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8249.321224] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8249.338507] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8253.286725] Lustre: server umount tfs-MDT0000 complete [ 8257.725362] LustreError: 246508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786185654 with bad export cookie 6076516885390887869 [ 8257.744696] LustreError: 246508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8257.744975] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8258.302623] Lustre: server umount tfs-MDT0001 complete [ 8273.159651] Lustre: server umount tfs-OST0000 complete [ 8275.999120] Lustre: 235055:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786185656/real 1786185656] req@ffff99951342a300 x1872950976531072/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786185672 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8276.050923] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8276.063165] Lustre: Skipped 3 previous similar messages [ 8277.404325] Lustre: server umount tfs-OST0001 complete [ 8299.901968] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 8310.152703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8310.650090] LustreError: 251798:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8310.680488] LustreError: 251798:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 6 previous similar messages [ 8310.738823] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8310.744770] Lustre: Skipped 3 previous similar messages [ 8315.182554] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8323.347326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8328.402243] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8331.141235] Lustre: 252938:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8336.993615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8343.474415] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8347.605174] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8354.456358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8355.897975] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8355.915549] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8360.992545] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8368.578552] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8372.481382] Lustre: 254810:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8379.529480] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:42:54 (1786185774) [ 8396.780583] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8396.795402] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8396.800304] Lustre: Skipped 2 previous similar messages [ 8396.832273] Lustre: Skipped 5 previous similar messages [ 8401.875225] Lustre: server umount lustre-MDT0000 complete [ 8406.883261] LustreError: 251774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786185803 with bad export cookie 6076516885390889794 [ 8406.892340] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8406.900609] LustreError: 251774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8421.343148] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8421.789797] Lustre: server umount lustre-MDT0001 complete [ 8440.287226] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8440.386984] Lustre: server umount lustre-OST0000 complete [ 8450.253605] Lustre: server umount lustre-OST0001 complete [ 8457.073766] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 8466.072944] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 8499.652071] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8499.652071] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8499.652071] [ 8499.690597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8500.003935] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8500.046819] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8500.143270] Lustre: lustre-MDT0000: new disk, initializing [ 8500.256167] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8500.259119] Lustre: Skipped 3 previous similar messages [ 8500.298415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8504.667298] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8514.095469] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8514.095469] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8514.095469] [ 8514.131134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8514.242738] Lustre: 257998:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8514.287514] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8514.291106] Lustre: Skipped 1 previous similar message [ 8514.367848] Lustre: lustre-MDT0001: new disk, initializing [ 8514.522334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8514.551573] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8519.015755] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8530.072901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8530.426455] Lustre: lustre-OST0000: new disk, initializing [ 8530.430061] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8530.440388] Lustre: 259581:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8532.279857] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8532.302525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8532.487179] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8538.059312] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8549.452877] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8549.690437] Lustre: lustre-OST0001: new disk, initializing [ 8549.694786] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8549.700575] Lustre: 260453:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8551.292825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8551.298437] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8551.419466] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8556.433056] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8567.288346] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:46:02 (1786185962) [ 8583.616268] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:46:18 (1786185978) [ 8602.600595] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8602.602611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8602.612269] Lustre: Skipped 4 previous similar messages [ 8602.616117] Lustre: Skipped 9 previous similar messages [ 8605.660279] Lustre: server umount lustre-OST0000 complete [ 8607.728056] LustreError: 259565:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8607.752944] LustreError: 259565:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 17 previous similar messages [ 8611.809365] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8616.558251] Lustre: server umount lustre-MDT0000 complete [ 8621.327464] LustreError: 257989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186017 with bad export cookie 6076516885390891565 [ 8621.342682] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8621.344747] LustreError: 257989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8627.845736] Lustre: server umount lustre-MDT0001 complete [ 8642.452445] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 8712.092920] Lustre: server umount lustre-OST0001 complete [ 8718.695034] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 8737.348403] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 8773.139030] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8773.500573] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8773.536239] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8773.666273] Lustre: lustre-MDT0000: new disk, initializing [ 8773.748127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8773.750599] Lustre: Skipped 3 previous similar messages [ 8773.769154] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8779.287856] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8792.684889] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8792.870646] Lustre: 265252:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8792.919954] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8792.924848] Lustre: Skipped 1 previous similar message [ 8793.006985] Lustre: lustre-MDT0001: new disk, initializing [ 8793.162315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8793.181643] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8798.307947] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8808.177969] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8814.506240] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8821.847797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8822.164903] Lustre: 267222:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8823.566136] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8823.703757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8829.778787] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8841.094879] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8849.377875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8849.392439] Lustre: Skipped 4 previous similar messages [ 8849.403312] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8849.405058] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8849.440497] Lustre: Skipped 7 previous similar messages [ 8853.909438] Lustre: server umount lustre-OST0000 complete [ 8859.627986] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8864.563621] Lustre: server umount lustre-MDT0000 complete [ 8864.748390] LustreError: 265263:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8864.775603] LustreError: 265263:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 8 previous similar messages [ 8868.209473] LustreError: 265243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186264 with bad export cookie 6076516885390893042 [ 8868.209772] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8868.216141] LustreError: 265243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8868.746170] Lustre: server umount lustre-MDT0001 complete [ 8880.993253] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 8883.531390] Key type lgssc unregistered [ 8883.788966] LNet: 268989:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8883.804960] LNetError: 268989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8883.822366] LNet: Removed LNI 192.168.202.128@tcp [ 8884.828409] Key type .llcrypt unregistered [ 8884.830675] Key type ._llcrypt unregistered [ 8912.836245] Key type ._llcrypt registered [ 8912.838597] Key type .llcrypt registered [ 8912.918068] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 8914.475078] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8914.512347] alg: No test for adler32 (adler32-zlib) [ 8915.634331] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 8915.837770] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 8917.535200] Key type lgssc registered [ 8918.726774] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8925.835859] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8925.847962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8927.237873] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8932.083990] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8937.446686] LustreError: 270452:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8937.476594] LustreError: 270452:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 8941.384724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8941.695730] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8945.746825] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8951.956930] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8956.331179] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8963.354567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8963.854295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8970.629511] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 8972.028820] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8979.511741] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8980.570069] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8987.682996] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8987.686044] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8987.699785] LustreError: Skipped 1 previous similar message [ 8987.709615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8987.727979] Lustre: Skipped 1 previous similar message [ 8987.752641] Lustre: Skipped 1 previous similar message [ 8992.759832] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8992.767431] Lustre: Skipped 1 previous similar message [ 8997.343173] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8997.743261] Lustre: server umount lustre-OST0000 complete [ 9002.977071] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9003.016423] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9008.102775] LustreError: 272291:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9008.125568] LustreError: 272291:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 9008.248440] Lustre: server umount lustre-MDT0000 complete [ 9012.055123] LustreError: 270431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186408 with bad export cookie 7198563504984844465 [ 9012.059242] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9012.070857] LustreError: 270431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9012.431360] Lustre: server umount lustre-MDT0001 complete [ 9022.527334] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 9025.316826] Key type lgssc unregistered [ 9025.674673] LNet: 274042:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9025.682850] LNetError: 274042:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9025.701293] LNet: Removed LNI 192.168.202.128@tcp [ 9026.477943] Key type .llcrypt unregistered [ 9026.483132] Key type ._llcrypt unregistered [ 9041.319967] Key type ._llcrypt registered [ 9041.321493] Key type .llcrypt registered [ 9041.416733] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 9042.335992] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9042.363599] alg: No test for adler32 (adler32-zlib) [ 9043.369397] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 9043.532224] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 9045.176151] Key type lgssc registered [ 9046.162742] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9053.056305] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9053.069725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9068.575978] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9073.695776] LustreError: 275237:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.128@tcp: failed processing log, type 4: rc = -110 [ 9100.387308] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9106.471379] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9114.243164] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9116.288799] Lustre: server umount lustre-OST0000 complete [ 9133.634240] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 9136.456498] Key type lgssc unregistered [ 9136.746079] LNet: 276872:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9136.757154] LNetError: 276872:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9136.792902] LNet: Removed LNI 192.168.202.128@tcp [ 9137.634580] Key type .llcrypt unregistered [ 9137.637973] Key type ._llcrypt unregistered [ 9173.430513] Key type ._llcrypt registered [ 9173.432568] Key type .llcrypt registered [ 9173.526782] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 9174.437687] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9174.574624] alg: No test for adler32 (adler32-zlib) [ 9175.699785] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 9175.929981] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 9177.581098] Key type lgssc registered [ 9178.578716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9189.811183] loop0: detected capacity change from 0 to 204800000 [ 9189.980889] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9189.990574] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9190.341042] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9191.198322] loop0: detected capacity change from 0 to 204800000 [ 9191.374538] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9191.386370] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9191.751748] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9192.675546] loop0: detected capacity change from 0 to 204800000 [ 9192.888508] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9192.900613] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9193.018357] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9194.130389] loop0: detected capacity change from 0 to 204800000 [ 9194.303475] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9195.330744] loop0: detected capacity change from 0 to 204800000 [ 9195.360559] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9198.901266] loop0: detected capacity change from 0 to 204800000 [ 9198.938090] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9202.304394] loop0: detected capacity change from 0 to 204800000 [ 9202.364255] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9206.061134] loop0: detected capacity change from 0 to 204800000 [ 9206.081163] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9210.074801] loop0: detected capacity change from 0 to 204800000 [ 9210.157741] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9210.173358] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9210.246871] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9210.253892] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9210.261469] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9210.270249] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9210.278205] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9210.286560] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9210.294591] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9210.303693] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9210.315052] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9210.320937] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9210.327863] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9210.335299] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9210.342558] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9210.349317] Lustre: 280291:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9215.897320] Lustre: server umount MGS complete [ 9216.727386] loop0: detected capacity change from 0 to 204800000 [ 9216.789270] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9217.166368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9217.209665] Lustre: lustre-MDT0000: Aborting client recovery [ 9217.225758] LustreError: 280666:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 9218.051883] loop1: detected capacity change from 0 to 204800000 [ 9218.150943] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9218.255288] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9218.271341] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9218.279546] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9218.286352] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9218.298338] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9218.306656] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9218.315218] Lustre: 280723:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9218.326747] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9218.456118] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9218.501175] Lustre: lustre-MDT0001: Aborting client recovery [ 9219.317685] loop2: detected capacity change from 0 to 204800000 [ 9219.441274] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9219.552347] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9219.568910] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9219.574912] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9219.597628] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9219.607379] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9219.618474] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9219.629094] Lustre: 280801:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9219.742750] Lustre: Found index 0 for lustre-OST0000, updating log [ 9219.763658] Lustre: 280820:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9219.823249] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9219.842151] Lustre: lustre-OST0000: Aborting recovery [ 9220.691184] loop3: detected capacity change from 0 to 204800000 [ 9220.796791] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9220.928662] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9220.935083] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9220.942542] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9220.952876] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9220.962838] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9220.976148] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9220.985530] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9221.004808] Lustre: Found index 1 for lustre-OST0001, updating log [ 9221.034780] Lustre: 280892:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9221.099568] Lustre: lustre-OST0001: Aborting recovery [ 9226.260265] LustreError: 280667:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 9227.248626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9227.323025] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9227.351554] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9227.455727] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9228.280714] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9228.295884] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9229.928630] LustreError: 280634:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 989us: rc = -2 [ 9230.445185] LustreError: 280635:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 495us: rc = -2 [ 9230.459921] LustreError: 280635:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 21 previous similar messages [ 9243.616565] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9243.620339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9243.630654] Lustre: Skipped 2 previous similar messages [ 9243.644784] Lustre: Skipped 3 previous similar messages [ 9248.739366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9248.751242] Lustre: Skipped 3 previous similar messages [ 9249.473507] Lustre: server umount lustre-MDT0000 complete [ 9250.643154] LustreError: 280616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186647 with bad export cookie 5819524352300337529 [ 9250.655085] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9250.663707] LustreError: 280616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9253.859797] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9253.861056] LustreError: 280632:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9253.882405] Lustre: Skipped 1 previous similar message [ 9253.908858] LustreError: 280632:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [ 9253.925607] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9255.912246] LustreError: 280633:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9255.941336] LustreError: 280633:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 9256.983518] Lustre: server umount lustre-MDT0001 complete [ 9272.287244] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9272.428775] Lustre: server umount lustre-OST0000 complete [ 9279.592874] Lustre: server umount lustre-OST0001 complete [ 9286.382547] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:58:02 (1786186682) [ 9310.174543] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 9320.247637] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 9360.134634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9360.370768] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9360.393531] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9360.448385] Lustre: lustre-MDT0000: new disk, initializing [ 9360.508071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9360.511656] Lustre: Skipped 1 previous similar message [ 9360.523114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9365.420231] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9376.120764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9376.254481] Lustre: 284961:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9376.313502] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9376.317621] Lustre: Skipped 1 previous similar message [ 9376.399231] Lustre: lustre-MDT0001: new disk, initializing [ 9376.524157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9376.564756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9376.577464] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9382.034396] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9392.066288] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9398.203749] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9405.966516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9406.206215] Lustre: lustre-OST0000: new disk, initializing [ 9406.209314] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9406.216080] Lustre: 286929:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9406.301818] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9411.654000] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9411.670960] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9411.791372] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9412.561873] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9422.017417] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9427.691970] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9442.013713] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9446.657853] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9451.222511] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9504.232910] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9504.241871] Lustre: Skipped 2 previous similar messages [ 9504.242097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9504.259127] Lustre: Skipped 3 previous similar messages [ 9508.645654] Lustre: server umount lustre-OST0000 complete [ 9514.465480] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9514.488197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9514.502663] Lustre: Skipped 1 previous similar message [ 9514.515663] Lustre: Skipped 1 previous similar message [ 9520.247258] Lustre: server umount lustre-MDT0000 complete [ 9523.681063] LustreError: 284972:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9523.702093] LustreError: 284972:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 9524.828490] LustreError: 284953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186921 with bad export cookie 5819524352300340224 [ 9524.850511] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9524.852915] LustreError: 284953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9525.253330] Lustre: server umount lustre-MDT0001 complete [ 9532.762707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9536.943090] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9540.940722] Lustre: server umount MGS complete [ 9551.601144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9551.969410] LustreError: 290808:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9551.989315] LustreError: 290821:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9552.027601] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9555.823882] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9557.479674] LustreError: 290808:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9557.493745] LustreError: 290808:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [ 9563.937152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9568.952886] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9576.220393] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9580.991397] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9588.590856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9589.148263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9589.157592] Lustre: Skipped 1 previous similar message [ 9595.575860] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9602.414215] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9636.324402] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9636.335342] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9636.351226] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9636.355984] Lustre: Skipped 3 previous similar messages [ 9638.369714] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9639.964095] Lustre: server umount lustre-OST0000 complete [ 9649.674093] Lustre: server umount lustre-MDT0000 complete [ 9653.741918] LustreError: 292097:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9653.780092] LustreError: 292097:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [ 9654.081762] LustreError: 293645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786187050 with bad export cookie 5819524352300342072 [ 9654.084287] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9654.090926] LustreError: 293645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9654.879914] Lustre: server umount lustre-MDT0001 complete [ 9669.512834] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [ 9672.802648] Key type lgssc unregistered [ 9673.101449] LNet: 294759:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9673.110333] LNetError: 294759:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9673.134821] LNet: Removed LNI 192.168.202.128@tcp [ 9674.578617] Key type .llcrypt unregistered [ 9674.582324] Key type ._llcrypt unregistered [ 9716.733697] Key type ._llcrypt registered [ 9716.743705] Key type .llcrypt registered [ 9716.913822] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [ 9732.389151] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [ 9733.328702] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9733.355060] alg: No test for adler32 (adler32-zlib) [ 9734.495344] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [ 9734.749612] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [ 9736.399751] Key type lgssc registered [ 9737.870474] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9776.028450] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9776.088235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9777.412499] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9777.444919] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9777.525524] Lustre: lustre-MDT0000: new disk, initializing [ 9777.625828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9777.696934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9782.992972] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9795.008905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9795.272916] Lustre: 298715:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9795.387456] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9795.391361] Lustre: Skipped 1 previous similar message [ 9795.573222] Lustre: lustre-MDT0001: new disk, initializing [ 9795.709541] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9795.782130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9795.809538] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9801.656935] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9813.072353] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9820.442573] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9828.771465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9828.995983] Lustre: lustre-OST0000: new disk, initializing [ 9828.998782] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9829.002883] Lustre: 300687:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9829.115267] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9836.236377] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9836.737223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9836.754735] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9836.887387] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9846.037921] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9853.362662] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9865.534331] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9875.681687] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9887.672244] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9934.307135] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9934.308271] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9934.309609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9934.323698] Lustre: Skipped 1 previous similar message [ 9934.369085] LustreError: Skipped 1 previous similar message [ 9938.826454] Lustre: server umount lustre-OST0000 complete [ 9944.034180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9944.044638] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9944.059051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9944.061762] Lustre: Skipped 1 previous similar message [ 9949.668071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9949.683429] Lustre: Skipped 2 previous similar messages [ 9950.007171] Lustre: server umount lustre-MDT0000 complete [ 9953.900691] LustreError: 299513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786187350 with bad export cookie 9964728720390745496 [ 9953.908502] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9953.913968] LustreError: 299513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9954.420798] Lustre: server umount lustre-MDT0001 complete [ 9963.377674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9968.612715] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9973.633579] Lustre: server umount MGS complete [ 9986.937356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9987.303424] LustreError: 304680:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9987.324367] LustreError: 304680:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [ 9987.333827] LustreError: 304693:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9987.390774] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9992.684698] LustreError: 304681:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9992.946636] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [ 9996.769692] LustreError: 304680:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10001.905969] LustreError: 304681:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10002.053836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10002.498782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10007.319367] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [10014.431877] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10020.741432] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10027.711560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10028.040034] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10034.230736] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [10042.124336] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10070.601935] Lustre: server umount lustre-OST0000 complete [10078.192249] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10078.192959] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10078.198373] Lustre: Skipped 2 previous similar messages [10078.215130] Lustre: Skipped 1 previous similar message [10083.297170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10083.307745] Lustre: Skipped 1 previous similar message [10084.457135] Lustre: server umount lustre-MDT0000 complete [10088.020504] LustreError: 307518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786187484 with bad export cookie 9964728720390747421 [10088.029405] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10088.045179] LustreError: 307518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10097.929689] Lustre: server umount lustre-MDT0001 complete [10110.503354] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [10113.569527] Key type lgssc unregistered [10113.886267] LNet: 308629:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10113.901246] LNetError: 308629:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10113.951492] LNet: Removed LNI 192.168.202.128@tcp [10114.868856] Key type .llcrypt unregistered [10114.870627] Key type ._llcrypt unregistered [10135.643914] Key type ._llcrypt registered [10135.647166] Key type .llcrypt registered [10135.776878] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [10150.078682] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [10151.842281] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10151.872974] alg: No test for adler32 (adler32-zlib) [10153.242513] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [10153.564615] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [10155.247262] Key type lgssc registered [10156.140686] Lustre: Echo OBD driver; http://www.lustre.org/ [10178.289754] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [10194.136490] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10194.160676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10195.500189] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10195.529658] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10195.609587] Lustre: lustre-MDT0000: new disk, initializing [10195.727318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10195.749162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10200.910745] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [10211.897343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10212.242114] Lustre: lustre-OST0000: new disk, initializing [10212.251393] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10212.257102] Lustre: Skipped 1 previous similar message [10212.262623] Lustre: 312682:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10212.368946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10218.724674] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [10220.188877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10220.202457] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10220.258746] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11042.282900] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11042.292311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11044.381286] Lustre: server umount lustre-MDT0000 complete [11048.453223] LustreError: 311869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188444 with bad export cookie 14131476803508458900 [11048.464751] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11048.481052] LustreError: 311869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11048.691520] Lustre: server umount lustre-OST0000 complete [11080.232870] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [11093.843647] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [11135.494795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11135.822863] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11135.864743] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11135.951478] Lustre: lustre-MDT0000: new disk, initializing [11136.021140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11136.042346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11141.965587] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11153.350729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11153.525606] Lustre: 317019:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11153.639255] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11153.648366] Lustre: Skipped 1 previous similar message [11153.764216] Lustre: lustre-MDT0001: new disk, initializing [11153.850218] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11153.888140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11153.894638] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11159.179624] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11169.837944] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11176.822665] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11185.285519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11185.847937] Lustre: lustre-OST0000: new disk, initializing [11185.852381] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11185.862912] Lustre: 318987:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11186.005208] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11192.922227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11192.931751] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11193.067056] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11193.843909] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11202.649307] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11207.140329] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11207.377540] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11212.378949] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11212.627265] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11218.913154] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11218.915896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11218.930688] LustreError: Skipped 1 previous similar message [11218.959880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11221.587138] Lustre: server umount lustre-OST0000 complete [11229.154334] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11229.173587] Lustre: Skipped 1 previous similar message [11229.187726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11229.199590] Lustre: Skipped 2 previous similar messages [11232.848277] Lustre: server umount lustre-MDT0000 complete [11236.411970] LustreError: 317012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188632 with bad export cookie 14131476803511399726 [11236.417695] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11236.426372] LustreError: 317012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11236.785794] Lustre: server umount lustre-MDT0001 complete [11240.225227] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11242.350950] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:30:37 (1786188637) [11248.602837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11248.987420] LustreError: 321299:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11249.006898] LustreError: 321299:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [11249.072607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11253.739223] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11254.243914] LustreError: 321299:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11254.262637] LustreError: 321299:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [11259.363120] LustreError: 321300:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11261.922716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11262.252587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11267.100879] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11275.546841] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11281.692227] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11288.615225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11288.945457] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11296.270530] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11304.049272] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11311.447465] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11311.670321] Lustre: lustre-OST0001: new disk, initializing [11311.673539] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11311.683886] Lustre: 324136:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11311.771737] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11317.886694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11317.904730] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11317.988607] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11319.008702] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11319.022434] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11319.325643] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11320.098401] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11320.116468] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11322.211304] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11322.237560] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11326.370389] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11326.388922] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11330.891202] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11331.204743] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11334.691852] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11334.722654] LustreError: 324429:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11337.912245] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9648c6426800.ost_server_uuid 50 [11339.933462] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9648c6426800.ost_server_uuid in FULL state after 0 sec [11346.497716] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9648c6426800.ost_server_uuid 50 [11348.154443] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9648c6426800.ost_server_uuid in IDLE state after 0 sec [11350.291663] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11365.874535] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11365.885310] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11365.892430] LustreError: Skipped 1 previous similar message [11365.933641] Lustre: Skipped 1 previous similar message [11365.946215] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11365.969474] Lustre: Skipped 1 previous similar message [11370.961885] Lustre: server umount lustre-OST0001 complete [11371.020581] LustreError: 323120:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11376.117610] LustreError: 325670:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11376.144956] LustreError: 325670:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [11381.220205] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11381.223315] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11381.243533] Lustre: Skipped 1 previous similar message [11381.251196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11381.253887] Lustre: Skipped 3 previous similar messages [11384.288270] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11384.816392] LustreError: 323120:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11384.825701] LustreError: 323120:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [11387.697091] Lustre: server umount lustre-OST0000 complete [11395.040361] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11395.054034] Lustre: Skipped 1 previous similar message [11395.062773] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11395.070934] Lustre: Skipped 3 previous similar messages [11398.619060] Lustre: server umount lustre-MDT0000 complete [11402.154797] LustreError: 321279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188798 with bad export cookie 14131476803511400601 [11402.158928] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11402.167372] LustreError: 321279:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11402.615827] Lustre: server umount lustre-MDT0001 complete [11416.416385] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [11420.032502] Key type lgssc unregistered [11420.305065] LNet: 326833:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11420.315214] LNetError: 326833:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11421.350421] LNet: Removed LNI 192.168.202.128@tcp [11422.211181] Key type .llcrypt unregistered [11422.213709] Key type ._llcrypt unregistered [11451.078906] Key type ._llcrypt registered [11451.087216] Key type .llcrypt registered [11451.261914] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [11452.608440] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11452.800151] alg: No test for adler32 (adler32-zlib) [11454.021566] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [11454.410967] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [11456.143976] Key type lgssc registered [11457.807774] Lustre: Echo OBD driver; http://www.lustre.org/ [11468.285527] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11468.339991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11469.971953] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11476.170557] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11480.560325] LustreError: 328255:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11480.591711] LustreError: 328255:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [11484.499816] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11485.668326] LustreError: 328256:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11490.784559] LustreError: 328255:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11491.796658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11492.336675] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11496.569619] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11507.417787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11507.839367] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11516.014601] LustreError: 329933:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11516.035139] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11516.334634] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11522.532326] LustreError: 329933:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11522.580060] LustreError: 329933:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [11526.792843] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11532.770660] LustreError: 329936:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11532.794599] LustreError: 329936:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [11534.413206] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11534.815591] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11539.976533] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11541.974580] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11550.172570] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11557.552423] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11557.962858] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11662.825785] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11662.830500] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11662.833115] LustreError: Skipped 1 previous similar message [11662.834903] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11662.848037] Lustre: Skipped 1 previous similar message [11662.854431] Lustre: Skipped 1 previous similar message [11666.997360] Lustre: server umount lustre-OST0001 complete [11667.944970] LustreError: 330320:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11667.980702] LustreError: 330320:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [11675.104444] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11675.114107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11675.140427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11676.129626] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11676.140281] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11676.159243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11678.184301] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11680.521483] Lustre: server umount lustre-OST0000 complete [11686.880563] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11686.904205] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11686.931583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11686.937278] Lustre: Skipped 1 previous similar message [11698.656458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11698.669500] Lustre: Skipped 5 previous similar messages [11699.167384] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11699.468453] Lustre: server umount lustre-MDT0000 complete [11702.753297] LustreError: 328256:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11702.771773] LustreError: 328256:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 5 previous similar messages [11703.922336] LustreError: 328237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189100 with bad export cookie 2298486429121612276 [11703.927636] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11703.936377] LustreError: 328237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11704.424878] Lustre: server umount lustre-MDT0001 complete [11721.647642] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [11725.738515] Key type lgssc unregistered [11726.288773] LNet: 333217:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11726.297335] LNetError: 333217:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11726.317798] LNet: Removed LNI 192.168.202.128@tcp [11727.714663] Key type .llcrypt unregistered [11727.716367] Key type ._llcrypt unregistered [11785.638488] Key type ._llcrypt registered [11785.640975] Key type .llcrypt registered [11785.853591] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [11787.713013] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11787.992945] alg: No test for adler32 (adler32-zlib) [11789.591303] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [11790.100067] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [11791.849676] Key type lgssc registered [11793.641165] Lustre: Echo OBD driver; http://www.lustre.org/ [11805.561575] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11805.585772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11807.117147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11813.509482] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11817.443797] LustreError: 335509:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11817.465478] LustreError: 335509:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [11822.582450] LustreError: 335510:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11823.500034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11823.872934] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11828.816579] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11831.880487] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11838.828513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11839.315508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11846.566909] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11848.548158] LustreError: 336999:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11851.648981] LustreError: 336999:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11851.695240] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11856.868922] LustreError: 336999:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11856.896759] LustreError: 336999:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [11857.179827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11857.633380] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11863.045718] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11865.165798] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11875.667904] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11882.928719] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11898.856562] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11898.867850] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11898.869241] Lustre: Skipped 3 previous similar messages [11898.889905] Lustre: Skipped 3 previous similar messages [11903.226413] Lustre: server umount lustre-MDT0000 complete [11903.971638] LustreError: 338529:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11903.983779] LustreError: 338529:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 3 previous similar messages [11907.415824] LustreError: 335491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189303 with bad export cookie 10581067450168339130 [11907.423871] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11907.431049] LustreError: 335491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11908.122214] Lustre: server umount lustre-MDT0001 complete [11922.661894] Lustre: server umount lustre-OST0000 complete [11924.959268] Lustre: 334774:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786189305/real 1786189305] req@ffff999506c8dc00 x1872955111966720/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786189321 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11924.994197] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11926.285389] Lustre: server umount lustre-OST0001 complete [11948.300472] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [11959.876386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11960.286112] LustreError: 340772:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11960.385118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11965.519343] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11974.547943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11974.920086] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11980.002371] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [11983.872890] Lustre: 341913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11993.195318] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11993.756185] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11999.851117] LustreError: 342266:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11999.884792] LustreError: 342266:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [12001.147410] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12006.419601] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12012.493167] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12018.200364] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12020.343330] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12029.004750] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12033.026824] Lustre: 343785:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12038.637973] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12038.641665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12038.649473] Lustre: Skipped 2 previous similar messages [12038.680341] Lustre: Skipped 3 previous similar messages [12043.751363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12044.480540] Lustre: server umount lustre-MDT0000 complete [12049.113616] LustreError: 340753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189445 with bad export cookie 10581067450168340831 [12049.143599] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12049.802437] Lustre: server umount lustre-MDT0001 complete [12065.281578] Lustre: server umount lustre-OST0000 complete [12079.141600] Lustre: server umount lustre-OST0001 complete [12083.808926] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12086.250261] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12088.618756] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:44:43 (1786189483) [12122.819629] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [12133.982184] loop0: detected capacity change from 0 to 1099511627776 [12134.862292] SGI XFS with ACLs, security attributes, quota, no debug enabled [12134.877824] *** VALIDATE xfs *** [12134.961059] XFS (loop0): Mounting V5 Filesystem [12135.025467] XFS (loop0): Ending clean mount [12136.542888] loop1: detected capacity change from 0 to 18691697672192 [12272.298788] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [12287.808298] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [12327.326917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12327.555428] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12327.596304] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12327.692859] Lustre: lustre-MDT0000: new disk, initializing [12327.783766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12327.788525] Lustre: Skipped 1 previous similar message [12327.807247] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12333.173349] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12345.368351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12345.551616] Lustre: 348972:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12345.634224] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12345.639465] Lustre: Skipped 1 previous similar message [12345.769404] Lustre: lustre-MDT0001: new disk, initializing [12345.992561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12346.012471] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12350.840025] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12362.954966] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12370.552397] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12380.939297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12381.392705] Lustre: lustre-OST0000: new disk, initializing [12381.397774] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12381.414724] Lustre: 350940:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12383.199375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12383.222490] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12383.360232] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12390.391982] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12403.515377] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12408.989986] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12409.356414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12415.923396] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12416.207688] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12419.071022] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12419.072082] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12419.084569] LustreError: Skipped 1 previous similar message [12419.086089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12419.086095] Lustre: Skipped 3 previous similar messages [12419.155290] Lustre: Skipped 2 previous similar messages [12424.166995] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12424.177903] Lustre: Skipped 2 previous similar messages [12433.375436] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12433.651656] Lustre: server umount lustre-OST0000 complete [12439.521611] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12439.525305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12439.544868] Lustre: Skipped 1 previous similar message [12439.555040] Lustre: Skipped 3 previous similar messages [12453.343244] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12453.805124] Lustre: server umount lustre-MDT0000 complete [12454.897751] LustreError: 348983:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12454.915645] LustreError: 348983:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 10 previous similar messages [12458.864071] LustreError: 348962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189855 with bad export cookie 10581067450168342385 [12458.889886] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12458.896441] LustreError: 348962:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [12459.282450] Lustre: server umount lustre-MDT0001 complete [12469.911029] XFS (loop0): Unmounting Filesystem [12477.140455] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:51:11 (1786189871) [12486.060602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12487.789603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12487.795975] Lustre: Skipped 2 previous similar messages [12493.482914] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12504.719236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12514.789749] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12524.211825] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12531.545493] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12542.185616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12552.128919] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12562.419740] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12573.687837] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12573.688996] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12573.694722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12573.694733] Lustre: Skipped 4 previous similar messages [12573.728105] Lustre: Skipped 1 previous similar message [12579.239083] Lustre: server umount lustre-OST0000 complete [12589.583509] Lustre: server umount lustre-MDT0000 complete [12594.150826] LustreError: 354234:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12594.179716] LustreError: 354234:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 5 previous similar messages [12594.255746] LustreError: 353487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189990 with bad export cookie 10581067450168343267 [12594.269771] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12594.600823] Lustre: server umount lustre-MDT0001 complete [12610.357891] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [12613.611473] Key type lgssc unregistered [12614.024731] LNet: 357141:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12614.030635] LNetError: 357141:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12615.079197] LNet: Removed LNI 192.168.202.128@tcp [12616.237772] Key type .llcrypt unregistered [12616.241410] Key type ._llcrypt unregistered [12650.172169] Key type ._llcrypt registered [12650.176433] Key type .llcrypt registered [12650.325401] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [12652.122154] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12652.153368] alg: No test for adler32 (adler32-zlib) [12653.189656] Lustre: Lustre: Build Version: 2.17.56_50_g80e3cdc [12653.404321] LNet: Added LNI 192.168.202.128@tcp [8/256/0/180] [12655.047495] Key type lgssc registered [12656.698892] Lustre: Echo OBD driver; http://www.lustre.org/ [12664.474993] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12664.502648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12666.079546] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12671.435851] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12675.555485] LustreError: 358710:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12675.576397] LustreError: 358710:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [12680.014909] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12680.572798] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12685.435533] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12693.925490] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12699.421245] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12706.904738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12707.385930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12713.971477] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12720.818383] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12731.873690] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12731.876390] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12731.895851] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12732.944494] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.28@tcp (stopping) [12734.950111] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12734.958107] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12737.633297] Lustre: server umount lustre-MDT0001 complete [12738.035101] LustreError: 358707:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12740.074527] LustreError: 358711:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12740.086163] LustreError: 358711:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 1 previous similar message [12743.154686] LustreError: 360451:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12748.279828] LustreError: 358705:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12748.304464] LustreError: 358705:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [12750.426268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12750.513245] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12750.518857] Lustre: Found index 1 for lustre-MDT0001, updating log [12750.562976] Lustre: 358698:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12750.899290] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12755.959619] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12755.989034] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [12759.049863] LustreError: 362300:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12759.061619] LustreError: 362300:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12759.089508] LustreError: 358702:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12759.179507] LustreError: 362314:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.128@tcp: cfg command failed: rc = -17 [12759.188560] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12759.188560] [13063.153474] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13063.164470] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13063.177717] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13063.195223] Lustre: Skipped 1 previous similar message [13063.199211] Lustre: Skipped 1 previous similar message [13068.258315] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13068.261025] Lustre: Skipped 1 previous similar message [13068.868518] Lustre: server umount lustre-MDT0001 complete [13070.868063] LustreError: 358705:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.28@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13070.895383] LustreError: 358705:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 2 previous similar messages [13082.446531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13082.620495] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13082.640681] Lustre: Found index 1 for lustre-MDT0001, updating log [13082.690868] Lustre: 358699:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13083.284615] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13087.862317] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [13088.322548] LustreError: 369961:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13088.344393] LustreError: 369961:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13088.412163] LustreError: 358702:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13088.425143] LustreError: 358702:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13088.532933] LustreError: 369969:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13088.551924] LustreError: 369969:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13088.565911] LustreError: 369969:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.128@tcp: cfg command failed: rc = -17 [13088.583848] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.128@tcp [13088.583848] [13088.766954] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13395.936246] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13395.938484] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13395.958278] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13395.990170] Lustre: Skipped 1 previous similar message [13400.679679] Lustre: server umount lustre-MDT0001 complete [13401.057442] LustreError: 363093:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13401.105600] LustreError: 363093:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 7 previous similar messages [13412.389591] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13412.476101] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13412.484602] Lustre: Found index 1 for lustre-MDT0001, updating log [13412.544607] Lustre: 358700:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13412.812347] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13417.978015] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13418.759456] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [13422.108906] LustreError: 378061:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13422.118447] LustreError: 378061:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13422.129397] LustreError: 358702:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13422.139975] LustreError: 358702:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13422.228439] LustreError: 378076:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13422.238818] LustreError: 378076:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.128@tcp: cfg command failed: rc = -114 [13422.248661] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13422.248661] [13740.518043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13740.521968] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13740.531723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13740.531733] Lustre: Skipped 2 previous similar messages [13740.557870] Lustre: Skipped 2 previous similar messages [13744.493477] Lustre: server umount lustre-MDT0000 complete [13745.637392] LustreError: 359439:0:(ldlm_lib.c:1189:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13745.660092] LustreError: 359439:0:(ldlm_lib.c:1189:target_handle_connect()) Skipped 8 previous similar messages [13747.839578] LustreError: 377657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191144 with bad export cookie 17870288895320113998 [13747.846605] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13747.853784] LustreError: 377657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13750.757049] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13754.551680] Lustre: server umount lustre-MDT0001 complete [13758.995816] Lustre: server umount lustre-OST0000 complete [13767.658226] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [13777.742362] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [13813.970174] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:13:29 (1786191209) [13820.827608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13821.050728] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13821.074473] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13821.129616] Lustre: lustre-MDT0000: new disk, initializing [13821.194498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13821.208893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13825.721280] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [13835.111187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13835.262184] Lustre: 388304:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13835.316987] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13835.321949] Lustre: Skipped 1 previous similar message [13835.432204] Lustre: lustre-MDT0001: new disk, initializing [13835.516265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13835.540692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13835.547803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13839.612278] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [13849.747509] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13854.410630] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13860.318512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13860.538568] Lustre: lustre-OST0000: new disk, initializing [13860.541145] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13860.545328] Lustre: 390270:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13860.617617] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13866.579669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13866.589134] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13866.689673] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13867.014302] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_default_debug -1 all [13876.579977] Lustre: DEBUG MARKER: oleg228-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13882.338706] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13882.344415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13882.353496] Lustre: Skipped 1 previous similar message [13882.363233] Lustre: Skipped 6 previous similar messages [13888.151467] Lustre: server umount lustre-MDT0000 complete [13891.286893] LustreError: 388297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191287 with bad export cookie 17870288895320115951 [13891.300535] LustreError: MGC192.168.202.128@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13891.303788] LustreError: 388297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13891.595077] Lustre: server umount lustre-MDT0001 complete [13905.291183] Lustre: server umount lustre-OST0000 complete [13929.820181] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing set_hostid [13938.930329] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing load_modules_local [13965.838602] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13655 sec ========= 08:16:01 (1786191361) [13967.171698] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 08:16:02 (1786191362) === [13969.857972] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 08:16:05 (1786191365) === [14003.120470] Lustre: DEBUG MARKER: oleg228-server.virtnet: executing unload_modules_local [14005.079302] Key type lgssc unregistered [14005.291859] LNet: 396638:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14005.295616] LNetError: 396638:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14005.305688] LNet: Removed LNI 192.168.202.128@tcp [14005.924103] Key type .llcrypt unregistered [14005.926207] Key type ._llcrypt unregistered